How does DLL data get mapped with different processes?
Leon
I was wondering how does static data in a DLL get mapped into
different process' address space?
For example let's say we have a static (global) variable, int x, and
some code to modify it in a dll, for example
"mov [x],10".
Then some process loads the dll, and OS relocates the offset of "x" in
the mov instruction to some value.
Then, another process loads the same dll, but offset of "x" can not be
relocated anymore, since first process already uses it. But yet that
same offset can not be used for this second process because, let's
say, that address is already in use by that process for some other
data.
So, how does windows manage to make the same code in a dll work for
multiple processes, and reference data specific to each process?
Thanks,
Leonid Portnoy
Raymond Chen
>the mov instruction to some value.
>
>Then, another process loads the same dll, but offset of "x" can not be
>relocated anymore, since first process already uses it.
Whether or not the first process is using the offset of "x" is
irrelevant since that's another process, hence a different
address space.
So when process 1 asks for "the memory at offset of x", it gets
process 1's copy of x. And when process 2 asks for "the memory
at offset of x", it gets process 2's copy of x. No conflict.
--
(My return address is intentionally invalid; delete ".---" to get my real address.
My responses are not to be considered official technical support or advice.)
Virgil Smith
>>the mov instruction to some value.
>>
>>Then, another process loads the same dll, but offset of "x" can not be
>>relocated anymore, since first process already uses it.
>
>Whether or not the first process is using the offset of "x" is
>irrelevant since that's another process, hence a different
>address space.
>
>So when process 1 asks for "the memory at offset of x", it gets
>process 1's copy of x. And when process 2 asks for "the memory
>at offset of x", it gets process 2's copy of x. No conflict.
You didn't answer his question. <None of the posts so far have, and
its a good question.>
Note that the poster is not confused about global vs. process local
memory. What the question was, was this...
1. DLL's get loaded into multiple processes' memory spaces.
2. For all processes there is one copy of the code and multiple copies
of the data. <presumably>
3. Upon "loading a process" the loader has to go "stomp" appropriate
offsets through the code <again presumably>.
Main Q: How is this accomplished?
Sub Q: Does the system keep track of loaded DLL's in such a way that
the loader can attempt to place all of the DLL code and data at the
same Virt Addresses for every process? <This seems extremely likely
otherwise the "one copy of the code" benefit of dynamic linking goes
right out the window.>
-----------== Posted via Newsfeeds.Com, Uncensored Usenet News ==----------
http://www.newsfeeds.com The Largest Usenet Servers in the World!
------== Over 73,000 Newsgroups - Including Dedicated Binaries Servers ==-----
Richard Masoner
Vir...@Nomadics.com (Virgil Smith) writes:
> You didn't answer his question. <None of the posts so far have, and
> its a good question.>
>
> Note that the poster is not confused about global vs. process local
> memory. What the question was, was this...
>
> 1. DLL's get loaded into multiple processes' memory spaces.
> 2. For all processes there is one copy of the code and multiple copies
> of the data. <presumably>
> 3. Upon "loading a process" the loader has to go "stomp" appropriate
> offsets through the code <again presumably>.
>
> Main Q: How is this accomplished?
>
> Sub Q: Does the system keep track of loaded DLL's in such a way that
> the loader can attempt to place all of the DLL code and data at the
> same Virt Addresses for every process?
Think "virtual memory." Your pointers to DLL global memory locations
are not pointers into physical memory, but virtual memory. The VM
system handles moving things around if necessary, but the actual
process is not modified, only the VM translation tables are changed.
For more details, read Chapter 3 of Viscarola and Mason's _Windows NT
Device Driver Development_.
*Caveat*: I'm much more familiar with UNIX internals than NT internals,
but presumably virtual memory is handled in much the same way.
Richard Masoner
Virgil Smith
question. Since, I've now received one attempt at a helpful answer by
email and one worthless reiteration of the answer I was trying to
steer the thread away from I obviously failed.
Please disregard my previous post.
Jose
pages containg the code. There can be a many-to-one relationship between
virtual pages & the physical memory that comprises the virtual addresses.
Unless the variable x is explicitly set to lie within a shared section of
the image that it is contained in (DLL, EXE, or otherwise), each process
mapping the image containing x will get its very own unique copy of x. The
pages backing X are marked as Copy-on-write by NT during load (when a
process attempts to write to x, a page fault occurs). So if you load the
DLL containig x in process A, & then use that same DLl in process B, the
physical memory that x actually resides in is shared by both processes until
one of them attempts to modify the contents of x, or any other variable
contained within the page that x resides in. When this happens, NT creates
changes the physical address that backs the virtual page containing x, to
another page-sized block of physical memory and copies the contents of the
old page to the newly allocated physical memory. The exact same thing
happens regardless of whether x has the same virtual address in process A
as it does in process B. Allocating the Page Table entries needed to
represent a virtual address is insignificant compared to the amount of
memory needed to acually back a virtual page.
If x lies within a shared section, then no fault implicit copy of x will
ever occur.
Virgil Smith wrote in message <37aee3bd...@216.65.40.40>...
>>>Then some process loads the dll, and OS relocates the offset of "x" in
>>>the mov instruction to some value.
>>>
>>>Then, another process loads the same dll, but offset of "x" can not be
>>>relocated anymore, since first process already uses it.
>>
>>Whether or not the first process is using the offset of "x" is
>>irrelevant since that's another process, hence a different
>>address space.
>>
>>So when process 1 asks for "the memory at offset of x", it gets
>>process 1's copy of x. And when process 2 asks for "the memory
>>at offset of x", it gets process 2's copy of x. No conflict.
>
>You didn't answer his question. <None of the posts so far have, and
>its a good question.>
>
>Note that the poster is not confused about global vs. process local
>memory. What the question was, was this...
>
>1. DLL's get loaded into multiple processes' memory spaces.
>2. For all processes there is one copy of the code and multiple copies
>of the data. <presumably>
>3. Upon "loading a process" the loader has to go "stomp" appropriate
>offsets through the code <again presumably>.
>
>Main Q: How is this accomplished?
>
>Sub Q: Does the system keep track of loaded DLL's in such a way that
>the loader can attempt to place all of the DLL code and data at the
>same Virt Addresses for every process? <This seems extremely likely
>otherwise the "one copy of the code" benefit of dynamic linking goes
>right out the window.>
>
>
Jerry Coffin
says...
> Hi,
>
> I was wondering how does static data in a DLL get mapped into
> different process' address space?
>
> For example let's say we have a static (global) variable, int x, and
> some code to modify it in a dll, for example
>
> "mov [x],10".
>
> the mov instruction to some value.
>
> Then, another process loads the same dll, but offset of "x" can not be
> same offset can not be used for this second process because, let's
> say, that address is already in use by that process for some other
> data.
>
> So, how does windows manage to make the same code in a dll work for
> multiple processes, and reference data specific to each process?
I'm probably going to include more detail than you need, but it looks
like the other articles you've gotten haven't entirely answered your
question.
An executable file (including a DLL) has a preferred load address.
Everything in the file (variables, procedures, etc.) are given
addresses starting from that preferred load address. Therefore, the
DLL you're dealing with might have a load address of (say) 0x10000000.
Items inside it are assigned addresses starting there, and just for
the sake of discussion, we'll say X ended up at the first address,
0x10000000.
When you start your program, the loader finds that it uses this DLL.
The loader looks at the addresses of the .EXE and other DLLs that have
already been loaded, and finds (for our discussion) that nothing else
is using the address range starting with 0x1000000, therefore it loads
this DLL starting at that address. Now, the variable X has an address
of 0x10000000 as well. Note, however, that this is a virtual address.
At any given time, the processor has a set of page tables that tell it
what physical addresses correspond to particular virtual addresses.
The OS maintains these tables and when you switch from one process to
another, it tells the processor the address of the page tables for the
new process.
Now, a typical DLL will have a number of sections, some for code, some
for variables, some for shared variables, etc. Each section has
attributes in the executable file. For example, code will be marked
as read-only, and data will be marked as read/write. When the loader
has mapped all of these to memory, it'll mark the pages containing
each one depending on the attributes of the section it's from. Intel
x86 processors normally use 4K pages of memory, so the linker normally
pads sections to 4K boundaries, so one section (and one set of
attributes) applies to a complete page of memory.
One of the section attributes that can be used is called Copy-On-Write
(COW). This actually sets the page of memory to read-only. Since the
memory is read-only, any attempt to write to the memory will raise an
exception. The OS exception handler then notes that the original
attribute was COW, so it makes a copy of that page for the process
that wrote to it. It then modifies the process's page table so that
the virtual address that was pointing to the original physical page
now points to the physical address of the new copy it just made. The
new page is then marked read/write, so the process can write to it
without causing an exception.
When/if another process loads the same DLL, it'll initially be given a
copy of a the pages that belong to the DLL itself -- I.e. the page
that's marked read-only. If the second process writes to the same
page, the OS will do the same thing again--make the second process a
copy of the page, and modify its page table so the virtual address it
expects now points to the second copy in physical memory.
Now, if a process has to load a DLL, but finds that the preferred
address of the DLL is already in use, it has to relocate the DLL.
This is done in a similar way: everywhere there's an address in the
code that refers to some other part of the DLL, the loader has to
overwrite the address with a modified address. E.g. if we had to
relocate the example DLL above to be based at 0x11000000, it would
find each reference to X, and change the code to use the address
0x11000000 instead of the 0x10000000 it has in the file. When it does
this, the same thing happens: a new copy is made of each page of the
DLL that gets modified, and the page tables for the process are
modified to point to the copy instead of the original code from the
DLL itself proper.
You can encounter a situation where one process might already have a
DLL loaded at the address of this DLL, and another doesn't. In this
case, the first one will make its own copy of the pages of the DLL
while the second will be able to simply map the DLL into its address
space and use them.
I'll also note that the explanation above assumes the OS is Windows
NT. Windows 95/98 do a few things a little bit differently, though
the effects are all pretty much the same -- if you went spelunking in
the Win9x kernel, you'd find a few things that don't quite fit the
explanation above.
Ole Nielsby
--------------------------------------------------------------------
NT tries to keep one copy of the stuff, mapped to the default
address. The mapping is calculated on a per process base, so,
when two processes load dlls in different order and their default
adresses collide, the dlls will be mapped differently.
If the stuff is changed - by loader fixups or by the running program
- NT will make extra copies of the affected pages (copy on write).
Data and program are treated the same.
---------------------------------------------------------------------
Win9x maps all dlls in one virtual address space, filling up the
system from 2G down. At times, it gets confused about it
and won't run anything.
Due to a design flaw in the 386 on which Win9x was designed
to run, the data segments need to be copied on access even if
they are not written to.
(Copy on write is implemented by fake write-protection. The
386 designers didn't know this, so they disabled write-protection
in kernel mode. The 486/P/PII has a flag to keep it enabled.)
---------------------------------------------------------------------
ON
Raymond Chen
<onie...@post4.tele.dk> wrote:
>Win9x maps all dlls in one virtual address space
More accurately, Win16 on Win9x maps all DLLs into one giant
address space. Win32 on Win9x uses separate address spaces.