Memory map of a Windows process
David Given
memory map looks like? For 32-bit.
I'm looking for things like where the main executable is loaded, what
areas are reserved for the heap, where any areas Windows itself uses
are, where DLLs appear, etc. All I can see on MSDN is that the bottom
two (or three) gigabytes are available for user space code, but it
doesn't say how this area is used.
What I'd really like to do is to clear off the entire bottom gigabyte or
so of address space for use with my own code; I have a plan in mind that
needs the use of fixed addresses, but I need to know what areas I can use...
--
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ───── http://www.cowlark.com ─────
│
│ 𝕻𝖍'𝖓𝖌𝖑𝖚𝖎 𝖒𝖌𝖑𝖜'𝖓𝖆𝖋𝖍 𝕮𝖙𝖍𝖚𝖑𝖍𝖚
│ 𝕽'𝖑𝖞𝖊𝖍 𝖜𝖌𝖆𝖍'𝖓𝖆𝖌𝖑 𝖋𝖍𝖙𝖆𝖌𝖓.
nico
> Does anyone know where I can find what a Windows' process internal
> memory map looks like? For 32-bit.
>
> I'm looking for things like where the main executable is loaded, what
> areas are reserved for the heap, where any areas Windows itself uses
> are, where DLLs appear, etc. All I can see on MSDN is that the bottom
> two (or three) gigabytes are available for user space code, but it
> doesn't say how this area is used.
It's managed by VMM :
http://msdn.microsoft.com/en-us/library/ms810616.aspx
David Given
[...]
Unfortunately that doesn't actually say how the user area is used --- it
merely describes (in commendable detail) how the VM manages the use
area, which isn't the same thing.
There is an interesting reference to the fact that system DLLs are
loaded into the kernel area rather than the user area... does this mean
that if I can avoid loading any user DLLs, I get the entire user area to
myself?
Jeroen Mostert
> Does anyone know where I can find what a Windows' process internal
> memory map looks like? For 32-bit.
>
> I'm looking for things like where the main executable is loaded, what
> areas are reserved for the heap, where any areas Windows itself uses
> are, where DLLs appear, etc.
I'm not aware of any such specific arrangement. Executables indicate their
own loading addresses, which will be respected barring a conflict, in which
case the executable will be remapped "somewhere else" -- where exactly
varies with what's already loaded, the version of Windows, the phase of the
moon, etc.
The only people who (should) count on this layout are the malware authors.
From Vista onwards, Windows implements address space layout randomization
(ASLR) where library addresses, heap allocations and stack addresses are
deliberately randomized to minimize the attack surface for buffer overflows,
making the notion of a fixed memory map obsolete in practice as well
(although libraries need to opt-in for it to work).
You can use VirtualQuery() to divine the current allocations, though not who
allocated them and why -- for that, you need to use tools like gflags and
WinDbg. This is usually reserved for troubleshooting memory leaks.
> What I'd really like to do is to clear off the entire bottom gigabyte or
> so of address space for use with my own code; I have a plan in mind that
> needs the use of fixed addresses, but I need to know what areas I can use...
>
I don't know what your "plan" is, but it doesn't sound good. At the very
least you could relax your assumptions to a gigabyte of contiguous address
space at *some* location, possibly rounded off to some multiple (other than
the system page size), and incorporate the offset either in all calculations
or through a fixup table. This is exactly how executables are remapped
(which use relative addressing to avoid the need for loading at an exact
address). The worst that could happen is that you lose some performance --
you'll live (probably).
Asking for a gigabyte of contiguous memory at a specific location is easy
enough; use VirtualAllocEx(). There is no address at which you're guaranteed
to get it, though. Even getting a gigabyte of contiguous memory somewhere
(specifying no particular base address) may be tricky enough with libraries
loading and allocating memory themselves. For 64-bit applications, of
course, this is practically a non-issue.
There may be a way of forcing Windows to reserve a gigabyte of memory as
part of a section in the PE file, but that's a guess. Someone who knows more
about the PE format than I may pipe up here. Even then you're probably not
going to get it at a specific address and the best that will happen is that
loading your executable will fail if the memory's not available.
--
J.
Alf P. Steinbach
> Does anyone know where I can find what a Windows' process internal
> memory map looks like? For 32-bit.
>
> I'm looking for things like where the main executable is loaded,
GetModuleHandle( 0 )
> what
> areas are reserved for the heap,
Heap functions?
> where any areas Windows itself uses
> are, where DLLs appear,
LoadLibrary (check if there's a GetLibrary)
> etc. All I can see on MSDN is that the bottom
> two (or three) gigabytes are available for user space code, but it
> doesn't say how this area is used.
>
> What I'd really like to do is to clear off the entire bottom gigabyte or
> so of address space for use with my own code; I have a plan in mind that
> needs the use of fixed addresses, but I need to know what areas I can use...
Uh, sounds like a bad idea.
If you describe what that would be *for*, then perhaps someone can suggest some
reasonable alternative.
Cheers & hth.,
- Alf
Bob Masta
<d...@cowlark.com> wrote:
>Does anyone know where I can find what a Windows' process internal
>memory map looks like? For 32-bit.
>
>I'm looking for things like where the main executable is loaded, what
>areas are reserved for the heap, where any areas Windows itself uses
>are, where DLLs appear, etc. All I can see on MSDN is that the bottom
>two (or three) gigabytes are available for user space code, but it
>doesn't say how this area is used.
>
>What I'd really like to do is to clear off the entire bottom gigabyte or
>so of address space for use with my own code; I have a plan in mind that
>needs the use of fixed addresses, but I need to know what areas I can use...
>
Keep in mind that there are no true "fixed
addresses" in Windows, at least as far as physical
memory access by user code. Everything is
virtualized and managed by Windows.
Every (normal) Windows app thinks it is loaded to
the same base address (0x00400000), but Windows
can map it to any part of physical memory it sees
fit...and change the location around at will,
including sending it to the hard drive if
something it deems more important needs the RAM
space.
This is all totally transparent to the user and
the coder. (Well, except in the case where the
hard drive has spun down to save power, and
suddenly an app needs something that was cached
there... might be a lag before it gets it.)
So, just allocate as much memory as you want,
using normal API calls. Windows will let you know
if it can't honor the request. Then just use the
memory normally. Don't worry, be happy!
Best regards,
Bob Masta
DAQARTA v5.00
Data AcQuisition And Real-Time Analysis
www.daqarta.com
Scope, Spectrum, Spectrogram, Sound Level Meter
Frequency Counter, FREE Signal Generator
Pitch Track, Pitch-to-MIDI
DaqMusic - FREE MUSIC, Forever!
(Some assembly required)
Science (and fun!) with your sound card!
David Given
[...]
> I don't know what your "plan" is, but it doesn't sound good. At the very
> least you could relax your assumptions to a gigabyte of contiguous
> address space at *some* location, possibly rounded off to some multiple
> (other than the system page size), and incorporate the offset either in
> all calculations or through a fixup table.
Unfortunately, I can't do that...
What I've got is a big pile of statically compiled code that's expecting
to be run at a particular address. It's not relocatable. I can use the
Window file mapping functions to page it in at the address it wants, but
I've got make sure that there's nothing there beforehand. If my
executable runs, attempts to map the code at the desired address, and I
discover that some DLL has been randomly put in the same place, then I'm
naturally out of luck.
Ideally want I want is to clear out everything below 0x40000000 or so,
so that *nothing* is mapped there, so that I can manage it all manually
myself. From what you've said there's nothing intrinsic to prevent this
happening --- Windows seems to be willing enough to put executables,
heap space etc anywhere it likes --- but I still need to be able to tell
it not to map anything below 0x40000000 until it's safe.
Jeroen Mostert
> On 13/02/10 12:19, Jeroen Mostert wrote:
> [...]
>> I don't know what your "plan" is, but it doesn't sound good. At the very
>> least you could relax your assumptions to a gigabyte of contiguous
>> address space at *some* location, possibly rounded off to some multiple
>> (other than the system page size), and incorporate the offset either in
>> all calculations or through a fixup table.
>
> Unfortunately, I can't do that...
>
> What I've got is a big pile of statically compiled code that's expecting
> to be run at a particular address. It's not relocatable.
Are we talking about Win32 code? Because that's pretty bogus.
> Ideally want I want is to clear out everything below 0x40000000 or so,
> so that *nothing* is mapped there, so that I can manage it all manually
> myself.
It's possible... but not pretty. Tell the linker that your executable's base
address is 0x10000 (you can't go lower than that) and fix it to that address
(/BASE:0x000010000 /FIXED). Now simply include a 1 GB array in your executable:
unsigned char bogus[0x40000000];
This will end up as your executable's .bss section, forcing the loader to
reserve at least 1 GB of memory. Keep in mind that this is your code plus
the statically compiled code plus the 1 GB for the bogus array -- making the
statically compiled code appear at the right address and not conflicting
with your own code is *your* problem. If you want to "load" the statically
compiled code (which would be copying it to memory), you will need to use
VirtualProtect() to mark the appropriate pages executable.
Beware that this gigabyte of memory is *committed* and counts towards the
system-wide commit limit. It's not possible to release or decommit this
memory using VirtualFree(), as it's considered part of the executable image.
If there is not 1 GB of physical memory available when your executable is
loaded to back these pages (where "physical" means RAM + swap), loading will
fail and/or the system will thrash growing the swap file.
For example, on my system, I can load two instances of such an application.
The third one will not load, as my system has only 4 GB of physical memory
(2 GB RAM and a statically sized 2 GB page file) and around a gigabyte is
already in use by saner applications combined. You should try to make this
range as small as your statically compiled code reasonably needs.
This is a pretty rude approach and I don't recommend it, but I also can
think of no other way of doing what you want. What I do recommend is taking
the statically compiled code out back and shooting it through the head, if
at all possible.
> From what you've said there's nothing intrinsic to prevent this happening
> --- Windows seems to be willing enough to put executables, heap space etc
> anywhere it likes --- but I still need to be able to tell it not to map
> anything below 0x40000000 until it's safe.
>
I know of no way of achieving that. ASLR notwithstanding, there is simply no
way to tell Windows where it cannot (or should) allocate the process heap or
the thread stacks, and those will be allocated before your code ever starts
to run. They will almost certainly be at a low address, barring dirty tricks
like the one above (where we do not prevent mapping there, we just make sure
only *we* are mapped there).
There is a registry setting that forces allocations in general to start from
the end of memory rather than the beginning (AllocationPreference, see
http://msdn.microsoft.com/library/bb613473) but I have no idea if it affects
the process heap or stacks. It's a global setting that affects more than
just your application, so it's not very appropriate to begin with.
--
J.
Sergei Zhirikov
> On 13/02/10 09:54, nico wrote:
> [...]
>> It's managed by VMM :
>> http://msdn.microsoft.com/en-us/library/ms810616.aspx
>
> Unfortunately that doesn't actually say how the user area is used --- it
> merely describes (in commendable detail) how the VM manages the use
> area, which isn't the same thing.
>
> There is an interesting reference to the fact that system DLLs are
> loaded into the kernel area rather than the user area... does this mean
> that if I can avoid loading any user DLLs, I get the entire user area to
> myself?
>
This in not true for Windows NT family.
David Given
[...]
>> What I've got is a big pile of statically compiled code that's expecting
>> to be run at a particular address. It's not relocatable.
>
> Are we talking about Win32 code? Because that's pretty bogus.
No, it's not --- that's the problem!
[...]
> unsigned char bogus[0x40000000];
That is, indeed, impressively evil.
I was rather hoping to be able to use MapViewOfFileEx(), and avoid
having to explicitly allocate and load the data, but of course you can
only map things into areas of memory that aren't currently in use... I
had never thought about just using brute force.
I'll keep that in mind, if all else fails. Thanks.
[...]
> I know of no way of achieving that. ASLR notwithstanding, there is
> simply no way to tell Windows where it cannot (or should) allocate the
> process heap or the thread stacks, and those will be allocated before
> your code ever starts to run. They will almost certainly be at a low
> address, barring dirty tricks like the one above (where we do not
> prevent mapping there, we just make sure only *we* are mapped there).
I've done some investigation of PE executables with objdump and I see
that they have a fairly standard section table:
Sections:
Idx Name Size VMA LMA File off Algn
0 .text 0001f154 00401000 00401000 00000400 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
1 .data 000000b0 00421000 00421000 0001f600 2**2
CONTENTS, ALLOC, LOAD, DATA
2 .rdata 00020e1c 00422000 00422000 0001f800 2**5
CONTENTS, ALLOC, LOAD, READONLY, DATA
3 .bss 00000d48 00443000 00443000 00000000 2**3
ALLOC
4 .idata 000013d8 00444000 00444000 00040800 2**2
CONTENTS, ALLOC, LOAD, DATA
5 .rsrc 0000b228 00446000 00446000 00041c00 2**2
CONTENTS, ALLOC, LOAD, DATA
It may be worth experimenting with adding a custom section.
Unfortunately the set of flags appear to be very poorly documented...
Leslie Milburn
"David Given" <d...@cowlark.com> wrote in message
news:hl7jjk$a9g$1...@news.eternal-september.org...
> On 14/02/10 00:33, Jeroen Mostert wrote:
> [...]
>>> What I've got is a big pile of statically compiled code that's expecting
>>> to be run at a particular address. It's not relocatable.
>>
>> Are we talking about Win32 code? Because that's pretty bogus.
>
> No, it's not --- that's the problem!
>
Well then it must be a port from the TRS80 or Commodore 64 because this sort
of logic is a load of garbage and is from the dark ages. Do you have the
source code because if you do it needs reworking and if you don't it needs
replacing.
Jeroen Mostert
[getting statically compiled code to appear at a particular address]
> I've done some investigation of PE executables with objdump and I see
> that they have a fairly standard section table:
>
> Sections:
> Idx Name Size VMA LMA File off Algn
> 0 .text 0001f154 00401000 00401000 00000400 2**4
> CONTENTS, ALLOC, LOAD, READONLY, CODE
> 1 .data 000000b0 00421000 00421000 0001f600 2**2
> CONTENTS, ALLOC, LOAD, DATA
> 2 .rdata 00020e1c 00422000 00422000 0001f800 2**5
> CONTENTS, ALLOC, LOAD, READONLY, DATA
> 3 .bss 00000d48 00443000 00443000 00000000 2**3
> ALLOC
> 4 .idata 000013d8 00444000 00444000 00040800 2**2
> CONTENTS, ALLOC, LOAD, DATA
> 5 .rsrc 0000b228 00446000 00446000 00041c00 2**2
> CONTENTS, ALLOC, LOAD, DATA
>
> It may be worth experimenting with adding a custom section.
Building a fixed-address DLL from the code and importing that may be even
more convenient than mucking with sections, but I don't know what format
your code is in.
If you do want to go the section route, you'll need tool support (you'll
probably need that for the DLL approach too). The linker that comes with
Visual Studio has no support for explicitly locating and sizing sections, as
far as I can tell. You can use #pragma section from within source files to
place variables in a custom section, but you cannot tell the linker to place
the contents of a particular .obj in a particular section.
You can use the mingw32 port of the GNU linker (ld), as it does have support
for explicit sections
(http://sourceware.org/binutils/docs-2.20/ld/Scripts.html). You can use
objcopy from the same toolchain to change the sections of the object file.
--
J.
Jonathan de Boyne Pollard
>
> Well then it must be a port from the TRS80 or Commodore 64 [...]
>
With 1GiB of RAM? That would have been some Commodore 64.