Tracking memory accesses
Marco Sambin
Is there a way of tracking memory accesses under Win NT ?
For instance, is it possible to know how many times a given application
has accessed a given memory location ??
Thanks to everybody.
Marco.
Tomas Restrepo
Slava,
> 4. Look for exceptions [WaitForDebugEvent() ] with code STATUS_GUARD_PAGE
> and the address belonging to the memory location. (STATUS_GUARD_PAGE is not
> defined in the include files (I wish I knew why); its numerical value is
> 0x80000001.)
Do you mean STATUS_GUARD_PAGE_VIOLATION? Its definition does appear on the Win2k
b3 SDK, and the VC6 headers . (Just a comment!)
#define STATUS_GUARD_PAGE_VIOLATION ((DWORD )0x80000001L)
Have fun
--
Tomas Restrepo
tom...@mvps.org
http://members.xoom.com/trestrep/
Slava M. Usov
Marco Sambin <sam...@tin.it> wrote in message
news:379ACE79...@tin.it...
> Is there a way of tracking memory accesses under Win NT ?
> For instance, is it possible to know how many times a given application
> has accessed a given memory location ??
A complex task, but still possible.
1. Run the application under (your own) debugger.
2. Use VirtualQueryEx() to get the current page protection of the memory
location in question.
3. Use VirtualProtectEx() to change protection of the memory location to
current_page_protection | PAGE_GUARD
4. Look for exceptions [WaitForDebugEvent() ] with code STATUS_GUARD_PAGE
and the address belonging to the memory location. (STATUS_GUARD_PAGE is not
defined in the include files (I wish I knew why); its numerical value is
0x80000001.)
5. Once you (your debugger) receive such an exception, do what you want to
do, then use SetThreadContext() to set the thread to single step execution,
then dispose of the debug event [ContinueDebugEvent() with DBG_CONTINUE]. If
the target process is multi-threaded, you should suspend all the other
threads (otherwise the other threads may access the memory location without
you seeing that).
6. Wait for EXCEPTION_SINGLE_STEP exception, after which call
ContinueDebugEvent() with DBG_CONTINUE and go to step 2. If you suspended
threads at the previous step, resume them now.
Pay attention to multi-threading issues, as outlined at steps 5 and 6.
Probably, you'll have to suspend/resume all threads at steps 2 and 3, also.
--
Slava
Please send any replies to this newsgroup.
Slava M. Usov
Tomas Restrepo <tom...@mvps.org> wrote in message
news:uKMgOLv1#GA.198@cppssbbsa03...
> Slava,
>
> > 4. Look for exceptions [WaitForDebugEvent() ] with code
STATUS_GUARD_PAGE
> > and the address belonging to the memory location. (STATUS_GUARD_PAGE is
not
> > defined in the include files (I wish I knew why); its numerical value is
> > 0x80000001.)
>
Win2k
> b3 SDK, and the VC6 headers . (Just a comment!)
> #define STATUS_GUARD_PAGE_VIOLATION ((DWORD )0x80000001L)
Thanks a lot, Tomas! So the includes weren't buggy; the docs were! (Fingers
are just itching to type "typic..".) Where they say STATUS_GUARD_PAGE, they
should STATUS_GUARD_PAGE_VIOLATION.
winnt.h
#define STATUS_GUARD_PAGE_VIOLATION ((DWORD )0x80000001L)
and winbase.h
#define EXCEPTION_GUARD_PAGE STATUS_GUARD_PAGE_VIOLATION
For readability, the switch after WaitForDebugEvent() should use
EXCEPTION_GUARD_PAGE, I believe...
Marco Sambin
Only a couple of things are still obscure to me: how could I set a thread to
single-step execution with SetThreadContext? I've analyzed the CONTEXT
structure, but I wouldn't know how to obtain this.
Moreover, when an instruction causes a STATUS_GUARD_PAGE_VIOLATION exception,
is it executed anyway or not ? In this latter case, how could I have it
executed after having revealed the memory access ?
Thank you very much,
Marco.
"Slava M. Usov" wrote:
> Marco Sambin <sam...@tin.it> wrote in message
> news:379ACE79...@tin.it...
> > Is there a way of tracking memory accesses under Win NT ?
> > For instance, is it possible to know how many times a given application
> > has accessed a given memory location ??
>
> A complex task, but still possible.
>
> 1. Run the application under (your own) debugger.
> 2. Use VirtualQueryEx() to get the current page protection of the memory
> location in question.
> 3. Use VirtualProtectEx() to change protection of the memory location to
> current_page_protection | PAGE_GUARD
> 4. Look for exceptions [WaitForDebugEvent() ] with code STATUS_GUARD_PAGE
> and the address belonging to the memory location. (STATUS_GUARD_PAGE is not
> defined in the include files (I wish I knew why); its numerical value is
> 0x80000001.)
> 5. Once you (your debugger) receive such an exception, do what you want to
> do, then use SetThreadContext() to set the thread to single step execution,
> then dispose of the debug event [ContinueDebugEvent() with DBG_CONTINUE]. If
> the target process is multi-threaded, you should suspend all the other
> threads (otherwise the other threads may access the memory location without
> you seeing that).
> 6. Wait for EXCEPTION_SINGLE_STEP exception, after which call
> ContinueDebugEvent() with DBG_CONTINUE and go to step 2. If you suspended
> threads at the previous step, resume them now.
>
> Pay attention to multi-threading issues, as outlined at steps 5 and 6.
> Probably, you'll have to suspend/resume all threads at steps 2 and 3, also.
>
Slava M. Usov
> Thanks a lot, Slava. You've been very clear and complete.
> Only a couple of things are still obscure to me: how could I set a thread
to
> single-step execution with SetThreadContext? I've analyzed the CONTEXT
> structure, but I wouldn't know how to obtain this.
Marco, forcing a thread to single step mode, as in general with manipulating
thread context, is platform-dependent. For i386 architecture (fancy name for
Intel processors) this is done through the flags register. Specifically,
call GetThreadContext(), and do pcontext->Eflags |= 0x100 (0x100, bit number
8 on, stands for single step trap flag); then call SetThreadContext().
> Moreover, when an instruction causes a STATUS_GUARD_PAGE_VIOLATION
exception,
> is it executed anyway or not ?
It's not executed. pcontext->EIP points exactly to the instruction that
caused the (guard page) exception. This is, BTW, different from the single
step exception, which is raised *after* the instruction was executed.
But you don't really have to do anything about pcontext->EIP.
> In this latter case, how could I have it
> executed after having revealed the memory access ?
Just call ContinueDebugEvent() with DBG_CONTINUE; this tells the OS that the
exception has been handled by the debugger.
Gary Nebbett
>
> Is there a way of tracking memory accesses under Win NT ?
> For instance, is it possible to know how many times a given application
> has accessed a given memory location ??
>
>
> Marco.
If you just want to track accesses to a few memory locations for debugging
purposes, you could do this with the cdb debugger - use debugger commands
like:
ba r1 77fcc3dc "r eip; g"
g
Here is the output of a sample debugging session:
w2k>cdb calc
Loaded dbghelp extension DLL
Loaded ntsdexts extension DLL
Microsoft(R) Windows 2000 Debugger
Version 5.00.2080.1
Copyright (C) Microsoft Corp. 1981-1999
CommandLine: calc
Symbol search path is: I:\winnt\symbols;I:\WINNT
CDB ModLoad: 01000000 01019000 calc.exe
CDB ModLoad: 77f80000 77ff8000 ntdll.dll
CDB ModLoad: 77640000 7787a000 I:\WINNT\system32\SHELL32.dll
CDB ModLoad: 77f40000 77f7b000 I:\WINNT\system32\GDI32.DLL
CDB ModLoad: 77e80000 77f33000 I:\WINNT\system32\KERNEL32.DLL
CDB ModLoad: 77e10000 77e72000 I:\WINNT\system32\USER32.DLL
CDB ModLoad: 77db0000 77e06000 I:\WINNT\system32\ADVAPI32.DLL
CDB ModLoad: 77d40000 77db0000 I:\WINNT\system32\RPCRT4.DLL
CDB ModLoad: 77cf0000 77d39000 I:\WINNT\system32\SHLWAPI.DLL
CDB ModLoad: 77bd0000 77c59000 I:\WINNT\system32\COMCTL32.DLL
CDB ModLoad: 78000000 78047000 I:\WINNT\system32\MSVCRT.dll
eax=00000000 ebx=00071f04 ecx=00000009 edx=00000000 esi=7ffdf000
edi=00071f70
eip=77f81360 esp=0006f984 ebp=0006fc98 iopl=0 nv up ei pl nz na pe
nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000
efl=00000202
ntdll!DbgBreakPoint:
77f81360 cc int 3
0:000> ba r1 77fcc3dc "r eip; g"
0:000> g
eip=77f97706
eip=77f8ee20
eip=77f8ed86
eip=77f8ed86
eip=77f8ed86
eip=77f8ed86
eip=77f8ed86
eip=77f8ed86
eip=77f8ed86
eip=77f8ed86
eip=77f8eeb8
eip=77f8eeb8
eip=77f8eeb8
eip=77f8eeb8
eip=77f8eeb8
eip=77f8eeb8
eip=77f86098
eip=77f86098
eip=77f86098
eip=77f87859
eip=77f87875
eip=77f86098
eip=77f87859
eip=77f87875
eip=77f87859
eip=77f87875
eip=77f87859
eip=77f87875
eip=77f87859
eip=77f87875
eip=77f87859
eip=77f87875
eip=77f87859
eip=77f87875
eip=77f87859
eip=77f87875
eip=77f8eeb8
eip=77f86098
eip=77f86098
eip=77f8eeb8
eip=77f86098
eip=77f87859
eip=77f87875
ntdll!ZwTerminateProcess+b:
77f91173 c20800 ret 0x8
0:000>
Now just count the lines.
Gary
Marco Sambin
If I may ask, I would have one more question: it looks like the single step
flag is reset every time a single step exception is raised. Is it correct ? If
I want to execute ALL my program in single-step mode, should I call
SetThreadContext every time I receive a SINGLE_STEP debug event, to restore the
single-step flag?
I've tried to do this (every time I receive a single step debug event, I set
the single step flag again), the program seems to execute step by step (I'm
logging the instruction pointer, too), but at a certain point I receive an
infinite (?) number of CreateThread debug events, each having a different
(progressing) Instruction Pointer (i.e. of the thread causing the CreateThread
debug event). Could you guess the reason for this behavior ?
Is there another way to count how many instructions have been execute before a
given debugging event than execute all the program in single step ??
Thank you for any answer,
Marco.
"Slava M. Usov" wrote:
> Marco Sambin <sam...@tin.it> wrote in message
Slava M. Usov
> Thanks a lot again Slava, I succeeded in putting my thread in single-step
mode.
>
> If I may ask, I would have one more question: it looks like the single
step
> flag is reset every time a single step exception is raised. Is it correct
? If
> I want to execute ALL my program in single-step mode, should I call
> SetThreadContext every time I receive a SINGLE_STEP debug event, to
restore the
> single-step flag?
Yes, this is correct. CPU resets the flag after the trap; otherwise, the
debugger itself would end up being in the single-step mode.
> I've tried to do this (every time I receive a single step debug event, I
set
> the single step flag again), the program seems to execute step by step
(I'm
> logging the instruction pointer, too), but at a certain point I receive an
> infinite (?) number of CreateThread debug events, each having a different
> (progressing) Instruction Pointer (i.e. of the thread causing the
CreateThread
> debug event). Could you guess the reason for this behavior ?
I once saw that, a program being debugged were spawning/killing threads
infinitely. Looks like a DLL does that for some reason. Sorry, I can't
suggest anything constructive.
> Is there another way to count how many instructions have been execute
before a
> given debugging event than execute all the program in single step ??
I don't think I understand what you mean. What do you want to do exactly?
Marco Sambin
I would like to count how many instructions the processor has executed from the
beginning of a program (process) to a given debugging event. I would like to
put this information together with the explanation of the debugging event, for
logging purposes (I'm doing a very particular kind of logging...).
Now my question is: is it possible to count how many instructions the processor
has executed in a way different from letting all program execute in single step
and, for instance, incresing a counter at every SINGLE_STEP exception ?
Clearer ?
Thanks a lot,
Marco.
Gary Nebbett
>
> I would like to count how many instructions the processor has executed from
the
> beginning of a program (process) to a given debugging event. I would like to
> put this information together with the explanation of the debugging event, for
> logging purposes (I'm doing a very particular kind of logging...).
> Now my question is: is it possible to count how many instructions the
processor
> has executed in a way different from letting all program execute in single
step
> and, for instance, incresing a counter at every SINGLE_STEP exception ?
Single stepping and counting is the only simple (albeit slow) method of
obtaining this information. Only the instructions executed in user mode are
counted.
Another possibility would be to instrument the debuggee, breaking its executable
code into basic blocks (sequences of code containing no branches) and counting
the execution of each block. For some RISC architectures there programs which
post-process existing executables to add this instrumentation. To obtain the
total number of instructions executed, multiply the invocation count of each
basic block by the number of instructions in the block.
For the Intel processor, you could obtain some approximate figures by using the
CPU performance monitoring counters. Depending on how you set the OS and USR
bits in the PerfEvtSel registers, you could count instructions executed in
user-mode, kernel-mode or both. I tried counting INST_DECODER and HW_INT_RX
events; if there are no HW_INT_RX events between two samples of INST_DECODER and
the section of code being counted does not invoke any system services which
might cause rescheduling then you can be fairly sure that no other process
contributed to the count.
To use these performance monitoring counters in your debugger scenario, the
debugger could read the counters when starting the debuggee and when the debug
event occurs. There would be some overhead because of the role of the smss and
csrss processes in the debugging architecture, but by measuring the overhead for
some known sequences of instructions, you could subtract this overhead from your
figures. You would also have to ensure that no other processes became ready
whilst counting.
A hybrid of all three methods would be to set the BTF flag in the DebugCtlMSR
register as well as the TF flag in the EFLAGS register, which would cause the
processor to single step on branches rather than on instructions.
Gary
Marco Sambin
I would like to ask you some additional questions:
I must specify that I'm working with Intel CPUs and under WinNT. Where could I find
information about the registers you were talking about in your answer and the
positions of the various flags ?
More specifically, does the DebugCtlMSR register correspond to the Dr7 register in
the CONTEXT structure ? What's the position of the BTF flag in this register ?
How could I access to the PerfEvtSel registers ?
Sorry for so many questions and thank you again,
Marco.
Marco Sambin
One last question, if I may ask: at the moment I'm only interested in having
single step exceptions on taken branches intead of on all instructions. Thanks
to the documentation you told me about, I've learned I can do it by putting the
value 1 in ECX and then calling the wrmsr instruction. The problem now is that
this instruction must be executed in privileged mode. Is there a way of
switching to privileged mode without writing a device driver ?
Furthermore, if I succeed in setting this flag, it's possible that the single
step exception will be caused by a process different from the one in which I
set the flag, isn't it ? (for instance, I set the flag, then my process
executes several instructions without any branches, then there is a context
switch and another process takes the CPU ownership: this latter process could
cause the single step exception at the first branch).
Isn't there a way of setting this flag in one process scope only ? Can't I set
it via the CONTEXT structure ?
Thanks again,
Jan Beulich
other process is doing debugging on the target system there would be no
ill effect. But you shouldn't really think of accessing MSRs without
having control of the system (ie without writing the OS kernel), and you
certainly can't access them (for good reason) without writing a driver.
--
-----------------------------
Jan Beulich
Novell Developer Support
email: DevSu...@Novell.com
Please, respond to news group messages only by another posting, not by
email.
Marco Sambin
the explanaton).
There is only one thing I would like some explanations about: by now, I've
decided to track memory accesses to the .data section of a very simple
executable (notepad). By analyzing the image of this executable, I obtain that
the .data section starts at address 0x01B68000 and it is 600 bytes long.
I've putted all this region in PAGE_GUARD mode with VirtualProtectEx().
When I receive PAGE_GUARD exceptions, I look at the ExceptionAddress field of
the debug event to look what's the address causing the violation: for the first
exceptions these address are feasable (0x01B68000, 0x01B68010, etc...), than I
start receiving PAGE_GUARD exceptions from memory regions that I didn't put to
PAGE_GUARD mode (0x77F8882F, etc...). They would seem addresses of code
sections...
I must specify that my test application (notepad) is a single-threaded
application and that normally it doesn't cause any PAGE_GUARD exception.
Thanks for the help.
Marco.
"Slava M. Usov" wrote:
> Marco Sambin <sam...@tin.it> wrote in message
> news:379ACE79...@tin.it...
> > Is there a way of tracking memory accesses under Win NT ?
> > For instance, is it possible to know how many times a given application
> > has accessed a given memory location ??
>
> A complex task, but still possible.
>
> 1. Run the application under (your own) debugger.
> 2. Use VirtualQueryEx() to get the current page protection of the memory
> location in question.
> 3. Use VirtualProtectEx() to change protection of the memory location to
> current_page_protection | PAGE_GUARD
> 4. Look for exceptions [WaitForDebugEvent() ] with code STATUS_GUARD_PAGE
> and the address belonging to the memory location. (STATUS_GUARD_PAGE is not
> defined in the include files (I wish I knew why); its numerical value is
> 0x80000001.)
> 5. Once you (your debugger) receive such an exception, do what you want to
> do, then use SetThreadContext() to set the thread to single step execution,
> then dispose of the debug event [ContinueDebugEvent() with DBG_CONTINUE]. If
> the target process is multi-threaded, you should suspend all the other
> threads (otherwise the other threads may access the memory location without
> you seeing that).
> 6. Wait for EXCEPTION_SINGLE_STEP exception, after which call
> ContinueDebugEvent() with DBG_CONTINUE and go to step 2. If you suspended
> threads at the previous step, resume them now.
>
> Pay attention to multi-threading issues, as outlined at steps 5 and 6.
> Probably, you'll have to suspend/resume all threads at steps 2 and 3, also.
>
Slava M. Usov
> I've made all this, and things seem to work pretty fine (many thanks again
for
> the explanaton).
> There is only one thing I would like some explanations about: by now, I've
> decided to track memory accesses to the .data section of a very simple
> executable (notepad). By analyzing the image of this executable, I obtain
that
> the .data section starts at address 0x01B68000 and it is 600 bytes long.
> I've putted all this region in PAGE_GUARD mode with VirtualProtectEx().
> When I receive PAGE_GUARD exceptions, I look at the ExceptionAddress field
of
> the debug event to look what's the address causing the violation: for the
first
> exceptions these address are feasable (0x01B68000, 0x01B68010, etc...),
than I
> start receiving PAGE_GUARD exceptions from memory regions that I didn't
put to
> PAGE_GUARD mode (0x77F8882F, etc...). They would seem addresses of code
> sections...
> I must specify that my test application (notepad) is a single-threaded
> application and that normally it doesn't cause any PAGE_GUARD exception.
I believe these are stack exceptions. Normally stack areas are framed by
guard pages.
Marco Sambin
pages these exceptions don't show up?
Thank you for any answer,
Marco.
Slava M. Usov
> But why if i run my test program under my debugger without setting the
guard
> pages these exceptions don't show up?
Sorry, no idea. The address you told about (0x77f8...) is address in
ntdll.dll...
Marco Sambin
a lot of problems and strange behaviors with this program. In particular:
- I'm avoiding the logging of memory accesses to memory regions that I don't
want to monitor (=that I didn't put to guard state) simply by checking at
logging time that the guard page exception comes from a desired memory region
(if not, I don't log it). But if I do this, I don't intercept any memory
access. Is it possible that a program (I've tried with Word and Notepad) never
accesses to its ".data" section pages ???
- Using notepad as test debuggee program, I've tried to put also the pages
belonging to the ".rdata" section to guard state. Even if VirtualProtectEx()
succeeds in putting to PAGE_GUARD state those pages, at a certain point the
debuggee program exits (while loading). It's a very strange behavior because
the debuggee program causes an EXIT_PROCESS debug event without causing a
previous second chance exception. I only receive a series of GUARD_PAGE
exceptions and then a sudden EXIT_PROCESS debug event, before notepad (= the
debuggee) gets loaded.
I really can't figure out what's happening...
And you ? Do you have any ideas ?
Thanks a lot to everybody for the help you're giving me.
Marco.
"Slava M. Usov" wrote:
> Marco Sambin <sam...@tin.it> wrote in message
> news:37A9A650...@tin.it...
> > I've made all this, and things seem to work pretty fine (many thanks again
> for
> > the explanaton).
> > There is only one thing I would like some explanations about: by now, I've
> > decided to track memory accesses to the .data section of a very simple
> > executable (notepad). By analyzing the image of this executable, I obtain
> that
> > the .data section starts at address 0x01B68000 and it is 600 bytes long.
> > I've putted all this region in PAGE_GUARD mode with VirtualProtectEx().
> > When I receive PAGE_GUARD exceptions, I look at the ExceptionAddress field
> of
> > the debug event to look what's the address causing the violation: for the
> first
> > exceptions these address are feasable (0x01B68000, 0x01B68010, etc...),
> than I
> > start receiving PAGE_GUARD exceptions from memory regions that I didn't
> put to
> > PAGE_GUARD mode (0x77F8882F, etc...). They would seem addresses of code
> > sections...
> > I must specify that my test application (notepad) is a single-threaded
> > application and that normally it doesn't cause any PAGE_GUARD exception.
>
> I believe these are stack exceptions. Normally stack areas are framed by
> guard pages.
>
Slava M. Usov
> Sorry for bothering all of you again with these subject, but I'm having
really
> a lot of problems and strange behaviors with this program. In particular:
> - I'm avoiding the logging of memory accesses to memory regions that I
don't
> want to monitor (=that I didn't put to guard state) simply by checking at
> logging time that the guard page exception comes from a desired memory
region
> (if not, I don't log it). But if I do this, I don't intercept any memory
> access. Is it possible that a program (I've tried with Word and Notepad)
never
> accesses to its ".data" section pages ???
Yes, quite possible. I'm not telling it happens *actually*. It may so that
you're setting guard pages too late, when the data have been accessed, or
too early, and the loader overwrites protection attributes...
> - Using notepad as test debuggee program, I've tried to put also the pages
> belonging to the ".rdata" section to guard state. Even if
VirtualProtectEx()
> succeeds in putting to PAGE_GUARD state those pages, at a certain point
the
> debuggee program exits (while loading). It's a very strange behavior
because
> the debuggee program causes an EXIT_PROCESS debug event without causing a
> previous second chance exception. I only receive a series of GUARD_PAGE
> exceptions and then a sudden EXIT_PROCESS debug event, before notepad (=
the
> debuggee) gets loaded.
It sounds like you're setting the guard pages before the sections have been
initialized by the loader. Which debug event do you use as a trigger? I'd
skip everything until the very first breakpoint came in... this is the point
when the loader has initialized the image and the kernel32.dll startup
routine is about to execute.
Marco Sambin
If I don't put the .rdata section's pages to guard mode, everything works.
Looking at the the addresses of the locations where GUARD_PAGE exceptions take
place, I've noticed that none of them belongs to the interval I set to guard
mode (.data section), but most of them are only a little bit less than the
.data section address interval. What section could they belong to ? I
understand that it's feasable that these addresses cause a page guard exception
because VirtualProtectEx changes the protection attributes of whole pages, not
of the exact interval I specified...
Furthermore, is there a way of knowing the address of the INSTRUCTION causing
the access violation ? (When I process the guard page exception, the Eip field
of the CONTEXT structure is the same as the Exception address...).
Thanks a lot,
Marco.
Slava M. Usov
> I'm using just the first breakpoint as trigger.
> If I don't put the .rdata section's pages to guard mode, everything works.
> Looking at the the addresses of the locations where GUARD_PAGE exceptions
take
> place, I've noticed that none of them belongs to the interval I set to
guard
> mode (.data section), but most of them are only a little bit less than
the
> .data section address interval. What section could they belong to ? I
First: if the image is produced by Microsoft tools, its section normally
follow each other without gaps (or with less-than-a-page gaps) , in the
following order:
.text
.rdata
.data
So if you're seeing addresses "a little bit less that the .data section",
it's .rdata; if .rdata is not present, it's .text.
Second: when you calculate the address of .data and .rdata section, do you
take into account the image base that the executable was *actually* mapped
at?
For example, my winword.exe says it prefers to be mapped at 0x30000000,
while notepad.exe at 0x1B40000, the default being 0x400000.
> understand that it's feasable that these addresses cause a page guard
exception
> because VirtualProtectEx changes the protection attributes of whole pages,
not
> of the exact interval I specified...
> Furthermore, is there a way of knowing the address of the INSTRUCTION
causing
> the access violation ? (When I process the guard page exception, the Eip
field
> of the CONTEXT structure is the same as the Exception address...).
Huh? The CONTEXT structure is what it is called, thread context. If it says
the thread's EIP is *here*, it is *here*. If
pde->Exception.ExceptionRecord.ExceptionAddress == pContext->Eip,
(pde being a pointer to the debug event, pContext a pointer to the thread's
context)
this indicates, *to me*, that the thread tried to execute an instruction at
an address protected by guard pages.
What are you trying to do anyway? Can you post code (if it's not huge) that
does it?
Marco Sambin
In your opinion, are the .data and .rdata sections good areas to monitor for my purpose ? Are there other meaningful areas ?
For what's concerning the problem while trying to monitor the .rdata section of notepad.exe (the program exits before getting loaded), I think that this is a consequence of the fact that undesired areas (0x77F6BC9F) take the guard attribute, even if I haven't called VirtualProtectEx on those areas. In fact, if ,from my debugger, I continue the guard page exceptions coming from those unwanted areas with DEBUG_EXCEPTION_NOT_HANDLED, I get a stack overflow exception and then an exit process debug event.
I really can't understand why those areas take the guard attribute and how to avoid this...
For what's concerning the code, I put hereafter the function that searches for data sections and changes their attributes to GUARD_PAGE (sorry if it's a little bit long...). As image base, I'm using the handle to the process as returned by CreateProcess(), called by the debugger (I'm receiving 0x01B60000 for notepad, 0x30000000 for Word, under NT4 SP4):
BOOL FindDataSections(HANDLE hProcess, PVOID PProcessBase)
{
   DWORD peHdrOffset;
   DWORD cBytesMoved;
   IMAGE_NT_HEADERS ntHdr;
   PIMAGE_SECTION_HEADER pSection;
   unsigned i;
   BOOL returnFlag = FALSE;
   char buffer[256];
   DWORD flOldProtect, flNewProtect;
   MEMORY_BASIC_INFORMATION mbi;
Â
   // Read in the offset of the PE header within
the debuggee
   if ( !ReadProcessMemory(ProcessInformation.hProcess,
                          Â
(PBYTE)PProcessBase + 0x3C,
                          Â
&peHdrOffset,
                          Â
sizeof(peHdrOffset),
                          Â
&cBytesMoved) )
       return FALSE;
Â
Â
   // Read in the IMAGE_NT_HEADERS.OptionalHeader.BaseOfCode
field
   if ( !ReadProcessMemory(ProcessInformation.hProcess,
                          Â
(PBYTE)PProcessBase + peHdrOffset,
                          Â
&ntHdr, sizeof(ntHdr), &cBytesMoved) )
       return FALSE;
   pSection = (PIMAGE_SECTION_HEADER)
              Â
((PBYTE)PProcessBase + peHdrOffset + 4
              Â
+ sizeof(ntHdr.FileHeader)
              Â
+ ntHdr.FileHeader.SizeOfOptionalHeader);
Â
   for ( i=0; i < ntHdr.FileHeader.NumberOfSections;
i++ )
   {
       IMAGE_SECTION_HEADER
section;
Â
       if ( !ReadProcessMemory(
ProcessInformation.hProcess,
                               Â
pSection, §ion, sizeof(section),
                               Â
&cBytesMoved) )
          Â
return FALSE;
Â
       // If it's the .data
(initialized data) section, get it
       if ( strncmp(section.Name,
".data", 5) == 0 )
       {
                      Â
sprintf(buffer,"Section .data found!\nVirtual address: %X\nSize: %d",
                                      Â
((DWORD)PProcessBase + section.VirtualAddress), section.SizeOfRawData );
                      Â
MessageBox(NULL,buffer,"Message",MB_OK);
Â
          Â
// Use this to get the current protection attributes
          Â
VirtualQueryEx( hProcess,
                          Â
(LPCVOID) ((DWORD)PProcessBase + section.VirtualAddress),
                          Â
&mbi,
                          Â
sizeof(mbi) );
          Â
// Take current attributes and add on the PageGuard flag
          Â
flNewProtect = mbi.Protect;
          Â
flNewProtect |= (PAGE_GUARD);
Â
          Â
if(VirtualProtectEx(hProcess,
                          Â
(LPVOID) ((DWORD)PProcessBase + section.VirtualAddress),
                          Â
section.SizeOfRawData,
                          Â
flNewProtect,
                          Â
&flOldProtect ))Â Â Â {
                              Â
DataSectionAddr = (PVOID) ((DWORD)PProcessBase + section.VirtualAddress);
                              Â
DataSectionSize = section.SizeOfRawData;
Â
                              Â
returnFlag = TRUE;
Â
                      Â
}
            else
                      Â
MessageBox(NULL,"VirtualProtectEx failed!","Message",MB_OK);
Â
Â
       }
Â
       else if ( strncmp(section.Name,
".rdata", 6) == 0 )
       {
          Â
sprintf(buffer,"Section .rdata found!\nVirtual address: %X\nSize: %d",
                                      Â
((DWORD)PProcessBase + section.VirtualAddress), section.SizeOfRawData );
          Â
MessageBox(NULL,buffer,"Message",MB_OK);
          Â
// Use this to get the current protection attributes
          Â
VirtualQueryEx( hProcess,
                          Â
(LPCVOID) ((DWORD)PProcessBase + section.VirtualAddress),
                          Â
&mbi,
                          Â
sizeof(mbi) );
          Â
// Take current attributes and add on the PageGuard flag
          Â
flNewProtect = mbi.Protect;
          Â
flNewProtect |= (PAGE_GUARD);
Â
          Â
if(VirtualProtectEx(hProcess,
                          Â
(LPVOID) ((DWORD)PProcessBase + section.VirtualAddress),
                          Â
section.SizeOfRawData,
                          Â
flNewProtect,
                          Â
&flOldProtect ))Â Â Â {
                              Â
RDataSectionAddr = (PVOID) ((DWORD)PProcessBase + section.VirtualAddress);
                              Â
RDataSectionSize = section.SizeOfRawData;
Â
                              Â
returnFlag = TRUE;
Â
                      Â
}
           else
                               MessageBox(NULL,"VirtualProtectEx failed!","Message",MB_OK);
              Â
}
Â
       pSection++; // Not this
section. Advance to next section.
   }
       return returnFlag;
}
This function is called upon reaching the first breakpoint.
If you are interested in helping me so deeply or you would like to
see other portions of my code, please tell me and I could send you the
project by e-mail (as an attachment).
Thank you very much for the interest you are demonstrating in my work and problems.
Marco.
Slava M. Usov
> What I'm *trying* to do is an utility that monitors the accesses to *data*
into
> memory: this is part of a larger project whose purpose is to monitor
various
> aspects of an application while running.
> In your opinion, are the .data and .rdata sections good areas to monitor
for my
> purpose ? Are there other meaningful areas ?
Well... these sections hold static data and the data that are used for
initialization of "real" variables (especially true for .rdata). The real
variables are those allocated on stack and on heap. So a better choice would
be to get those, although it's hard to accomplish (in particular, I don't
know how to handle data allocated on the heap). I think that trying to
monitor .rdata is rather wasteful, as the data are only read once from
there; .data is another story...
If you could get the list of the per-thread stack location/size data, that
would be great. However, how to get it is not documented anywhere. One vague
mention of it is in the descriptions of CREATE_PROCESS_DEBUG_INFO and
CREATE_THREAD_DEBUG_INFO structures, field lpThreadLocalBase.
There is also a definition of NT_TIB in winnt.h, which seems to be just what
is needed, but, still, it is not clear how to get it...
I believe this is information that MS make available to compiler writers,
but I may only guess. I'd like to know that myself! (Felix, if you're reding
this, what do you think?)
> For what's concerning the problem while trying to monitor the .rdata
section of
> notepad.exe (the program exits before getting loaded), I think that this
is a
> consequence of the fact that undesired areas (0x77F6BC9F) take the guard
> attribute, even if I haven't called VirtualProtectEx on those areas. In
fact,
> if ,from my debugger, I continue the guard page exceptions coming from
those
> unwanted areas with DEBUG_EXCEPTION_NOT_HANDLED, I get a stack overflow
> exception and then an exit process debug event.
> I really can't understand why those areas take the guard attribute and how
to
> avoid this...
This is weird. I don't know what's happening, sorry.
> For what's concerning the code, I put hereafter the function that searches
for
> data sections and changes their attributes to GUARD_PAGE (sorry if it's a
> little bit long...). As image base, I'm using the handle to the process as
> returned by CreateProcess(), called by the debugger (I'm receiving
0x01B60000
> for notepad, 0x30000000 for Word, under NT4 SP4):
Hm.. do you mean that you use the pi.hProcess as the image base [pi is a
PROCESS_INFORMATION structure returned by CreateProcess()]? If so, it's
*not* what you should do. Look at CREATE_PROCESS_DEBUG_INFO, field
lpBaseOfImage. However... you were able to get correct pointers to the
sections anyway... I believe I misunderstood you...
Marco Sambin
I received page guard exceptions from unwanted memory areas because it seems
that both pContext->Eip (as it's meaningful) and
...ExceptionRecord.ExceptionAddress (as it's NOT meaningful, to me) give the
address of the INSTRUCTION causing the exception. So, I was logging the
addresses of the instructions accessing to the data residing in memoy regions
that I protected (In fact, as you suggested me, they were addresses belonging
to the .text section).
To get the address of the real data causing the page guard exception, you must
look at ...ExceptionRecord.ExceptionInformation[1] field (this behavior is
documented for the Access Violation exception, but not for the PAGE_GUARD
exception: I discovered it by chance !!).
I still can't understand why protecting the .rdata section in notepad crashes
the program, but if this section is not meaningful...
> Hm.. do you mean that you use the pi.hProcess as the image base [pi is a
> PROCESS_INFORMATION structure returned by CreateProcess()]? If so, it's
> *not* what you should do. Look at CREATE_PROCESS_DEBUG_INFO, field
> lpBaseOfImage. However... you were able to get correct pointers to the
> sections anyway... I believe I misunderstood you..
Sorry, I was wrong while writing you that... Yes, I'm using lpBaseOfImage...
Thanks for all the help.
If I discover something more, I will let you know, if you are interested in
it...
Marco Sambin.
Slava M. Usov
> I have an answer to some of the problems I mentioned:
> I received page guard exceptions from unwanted memory areas because it
seems
> that both pContext->Eip (as it's meaningful) and
> ...ExceptionRecord.ExceptionAddress (as it's NOT meaningful, to me) give
> address of the INSTRUCTION causing the exception. So, I was logging the
> addresses of the instructions accessing to the data residing in memoy
regions
> that I protected (In fact, as you suggested me, they were addresses
belonging
> to the .text section).
> To get the address of the real data causing the page guard exception, you
must
> look at ...ExceptionRecord.ExceptionInformation[1] field (this behavior is
> documented for the Access Violation exception, but not for the PAGE_GUARD
> exception: I discovered it by chance !!).
Argh, good you found that out! Admittedly I was confused about this one and
made you confused too. Now that I *know* that the ExceptionAddress is the
address of the *instruction* that has caused the exception, it's only
natural: some of the exception codes (like EXCEPTION_PRIV_INSTRUCTION)
specify that the problem is the code, not data... so we should know the
instruction address, too.
> I still can't understand why protecting the .rdata section in notepad
crashes
> the program, but if this section is not meaningful...
Ahem... I should have noticed that earlier... this section is normally
merged with .idata section and, as such, contains IAT... for executables
that support delay-load, the delay import directory might be placed there,
as well. Simply put, this section is for system purpose only and does not
contain application data. That the IAT is there explains why you start
getting all sorts of guard exceptions: each time an API is called, a guard
exception is generated. As you don't handle it, it gets delivered to the
application and the application is terminated (you don't see the second
chance exceptions because the Visual C++ places main() into a try-except
block, which exits the program unconditionally).
> Thanks for all the help.
> If I discover something more, I will let you know, if you are interested
in
> it...
Nevermind. You're doing what I'd like to do myself, but am scared of it. I'm
very interested in the progress you are/will be making.
Gary Nebbett
>What I'm *trying* to do is an utility that monitors the accesses to *data*
> into memory: this is part of a larger project whose purpose is to monitor
>various aspects of an application while running.
>In your opinion, are the .data and .rdata sections good areas to monitor for my
>purpose ? Are there other meaningful areas ?
>
>
>If you are interested in helping me so deeply or you would like to see other portions
>of my code, please tell me and I could send you the project by e-mail (as an attachment).
You might find (depending on the application) that approaching 50% of all the instructions
access data. If you want to monitor all accesses to memory (which are not instruction
fetches), I think that your best bet would be to single step the application and analyze
each machine instruction. If you can be more specific about what you want to do, you
could mail me your application.
Gary
Marco Sambin
As you can see in my previous message on this newsgroup, I solved the main problems I had.
I would like to know some additional information about the following:
- How to locate the stack position and size (my idea would be to the retrieve the stack
pointer, make a VirtualQueryEx on that location, then retrieve size of the region from the
MEMORY_INFORMATION structure that is filled by VirtualQueryEx: is this meaningful ??).
- How to locate the heap(s).
Can you give me any suggestions about that ?
I think that single-stepping on each instruction would be a too high overhead (I've only
tried to count the executed instruction in this way and the overhead was already about
4000-5000%...).
To reduce a little bit the number of intercepted memory accesses, I could monitor only
certain stack locations or the heap only...
Thanks for your help,
Marco.
Gary Nebbett wrote:
> Marco Sambin <sam...@tin.it> wrote in message news:37B98A8C...@tin.it...
> >What I'm *trying* to do is an utility that monitors the accesses to *data*
> > into memory: this is part of a larger project whose purpose is to monitor
> >various aspects of an application while running.
> >In your opinion, are the .data and .rdata sections good areas to monitor for my
> >purpose ? Are there other meaningful areas ?
> >
> >[...]
> >
> >If you are interested in helping me so deeply or you would like to see other portions
> >of my code, please tell me and I could send you the project by e-mail (as an attachment).
>
Marco Sambin
I would have only a couple of questions: is there a way to understand to how
many bytes a memory read or write operation has accessed?
I explain better: when I intercept a memory access, I can know the address to
which I've accessed and the address of the instruction causing that access, but
I can't know how many bytes have been accessed (the "length" of the access). It
can't be more than a DWORD (32 bits), isn't it?
Another similar question: if we imagine the memory as composed of DWORD-aligned
DWORDs, could ONE memory read or write operation access to bytes belonging to
different DWORDs ? Do accesses to DWORDs have to be to DWORD-aligned locations
?
Thanks a lot,
Marco.
"Slava M. Usov" wrote:
> Marco Sambin <sam...@tin.it> wrote in message
> news:37BA71A9...@tin.it...
> > I have an answer to some of the problems I mentioned:
> > I received page guard exceptions from unwanted memory areas because it
> seems
> > that both pContext->Eip (as it's meaningful) and
> > ...ExceptionRecord.ExceptionAddress (as it's NOT meaningful, to me) give
> the
Slava M. Usov
Hi Marco. Good to hear from you.
> My memory accesses tracker is now working pretty fine.
I've been thinking on how to found out per-thread stack locations... there
is a structure NT_TIB defined in winnt.h. One big problem is how to find out
where it's located in the process memory space...
On x86, the TIB of the current thread is always at FS:0. To translate it to
the virtual address, one needs to do the following:
1. Get thread's context.
2. Get the value of FS (this is the number of the selector).
3. Call GetThreadSelector() against the selector.
LDF_ENTRY ldte;
GetThreadSelector(thread, FS, &ldte);
DWORD base_address = ldte.HighWord.Bytes.BaseHi << 24 +
ldte.HighWord.Bytes.BaseMid << 16 +
ldte.BaseLow;
// I hope I got that right :-)
NT_TIB *tib = (NT_TIB*)base_address;
if everything went okay, you should see that
tib == tib->Self
Do that for every thread that you can see, and you'll have TIBs of them all,
and, thus, their stack locations and sizes.
One word of warning: thread stacks *are* protected by guard pages, and the
faults, I believe, are handled either by the kernel or by the Win32
subsystem. I don't know what may happen if you plant a guard page on a
stack. If you do, please post your results.
> I would have only a couple of questions: is there a way to understand to
how
> many bytes a memory read or write operation has accessed?
> I explain better: when I intercept a memory access, I can know the address
to
> which I've accessed and the address of the instruction causing that
access, but
> I can't know how many bytes have been accessed (the "length" of the
access). It
> can't be more than a DWORD (32 bits), isn't it?
No, it cannot on x86. But may be one, two, or four bytes. But you'll have to
decode the instruction to figure out the "width" of the access.
> Another similar question: if we imagine the memory as composed of
DWORD-aligned
> DWORDs, could ONE memory read or write operation access to bytes belonging
to
> different DWORDs ? Do accesses to DWORDs have to be to DWORD-aligned
locations
> ?
This is *preferable*. On RISC machines this is a requirement (although the
OS may be instructed to work around itm at the expense of reduced
performance -- see SetErrorMode(), flag SEM_NOALIGNMENTFAULTEXCEPT for
deyails). On x86, however, anything is possible. Normally compilers produce
DWORD aligned code, though. But nothing prevents a programmer from doing
DWORD x = *((DWORD*)(0x10001));
The access to the data will be somewhat slower (because more than one read
bus cycle will be necessary), but still perfectly legitimate...
Marco Sambin
Could you only tell me what TIB stands for and what the FS segment register is
normally used for ?
I will let you know something as soon as I will try that.
Regards,
Marco.
"Slava M. Usov" wrote:
> Marco Sambin <sam...@tin.it> wrote in message
Slava M. Usov
> Hi Slava. Thanks a lot again.
> Could you only tell me what TIB stands for and what the FS segment
register is
> normally used for ?
From winnt.h:
typedef struct _NT_TIB {
struct _EXCEPTION_REGISTRATION_RECORD *ExceptionList;
PVOID StackBase;
PVOID StackLimit;
PVOID SubSystemTib;
union {
PVOID FiberData;
DWORD Version;
};
PVOID ArbitraryUserPointer;
struct _NT_TIB *Self;
} NT_TIB;
TIB is an acronym for Thread Information Block. There is another acronym,
TEB (Thread Environment Block) which seems to be a synonym, as follows from
#define GetCurrentFiber() (((PNT_TIB)NtCurrentTeb())->FiberData)
in the same header for ALPHA and PPC architectures. NtCurrentTeb() is
exported from ntdll.dll for every architecture, but the current TIB/TEB may
be determined without calling this function. For PPC, for example, in
winnt.h we can see:
// The address of the TEB is placed into GPR 13 at context switch time
// and should never be destroyed. To get the address of the TEB use
// the compiler intrinsic to access it directly from GPR 13.
and then
unsigned __gregister_get( unsigned const regnum );
#define NtCurrentTeb() ((struct _TEB *)__gregister_get(13))
For MIPS:
#define GetCurrentFiber() ((*(PNT_TIB *)0x7ffff4a8)->FiberData)
which seems to imply that TIB/TEB is placed at 0x7ffff4a8 (on MIPS, mind
you).
For ALPHA I found nothing like that, except that a compiler intrinsic
function is used:
#define NtCurrentTeb() ((struct _TEB *)_rdteb())
For x86, the header defines:
_inline PVOID GetFiberData( void ) { __asm {
mov eax, fs:[0x10]
mov eax,[eax]
}
}
_inline PVOID GetCurrentFiber( void ) { __asm mov eax, fs:[0x10] }
which indicates that the TIB/TEB is always at FS:[0] (0x10 is the offset of
FiberData in NT_TIB structure).
Thus the FS register is used as a convenient way to point to the TIB/TEB.
Given that among other things the TIB/TEB specifies are the base and the
limit of the thread stack, this structure may be very useful for you.
> I will let you know something as soon as I will try that.
Thanks.
Marco Sambin
I'm facing a new problem: I would like to understand not only where and when
the memory access has been done, but also what has been written or read. To
accomplish this, every time I receive a GUARD_PAGE exception, I save the
address of the memory access in the following field of a structure:
LastAccess.address; // it's a DWORD field
Then, when I receive the immediately following SINGLE_STEP exception, I go
watching what has been written or read by:
ReadProcessMemory ( lpDbgProcess->hProcess,
(LPVOID) LastAccess.address,
&MemData, // a DWORD
sizeof(MemData),
&nRead);
This works quite fine with simple programs. The problems come when trying with
complex ones, e.g. Microsoft Word. In particular, I've problems with the
addresses belonging to the .tls section: ReadProcessMemory() fails with error
code 998 ("Invalid access to memory location"). Even trying to call
VirtualProtectEx before ReadProcessMemory gives problems: it fails with error
87 ("The parameter is incorrect").
Do you know the reason for that ? Why .tls gives problems while .data works
pretty fine ?
Thank you very much,
Marco
"Slava M. Usov" wrote:
> Marco Sambin <sam...@tin.it> wrote in message
Slava M. Usov
> Thank you for the complete explanation.
>
> I'm facing a new problem: I would like to understand not only where and
when
> the memory access has been done, but also what has been written or read.
To
> accomplish this, every time I receive a GUARD_PAGE exception, I save the
> address of the memory access in the following field of a structure:
>
> LastAccess.address; // it's a DWORD field
>
> Then, when I receive the immediately following SINGLE_STEP exception, I go
> watching what has been written or read by:
>
> ReadProcessMemory ( lpDbgProcess->hProcess,
> (LPVOID) LastAccess.address,
>
> &MemData, // a DWORD
> sizeof(MemData),
> &nRead);
>
> This works quite fine with simple programs. The problems come when trying
with
> complex ones, e.g. Microsoft Word. In particular, I've problems with the
> addresses belonging to the .tls section: ReadProcessMemory() fails with
error
> code 998 ("Invalid access to memory location"). Even trying to call
> VirtualProtectEx before ReadProcessMemory gives problems: it fails with
error
> 87 ("The parameter is incorrect").
A remark not related to .tls: are you sure that the address you're stuffing
in LastAccess.address is DWORD aligned?
Then, what does VirtualQueryEx() say on this address?
> Do you know the reason for that ? Why .tls gives problems while .data
works
> pretty fine ?
I once read a vague description of .tls specifics; still I don't think I
have a complete understanding of how this works. If anyone can comment on
this, it will be highly appreciated.
From what I've heard, the .tls provides a per-thread mapping to VM. That is,
different threads get different views of the same VM area. Much like to how
separate processes have private address spaces. It may be, therefore, that
access to this area makes sense only in context of a particular thread...
obviously, this condition does not hold for a debugger (OS can't map the
"virtualized" address to a physical page because it can't identify the
thread for which the address is being virtualized, due to the "thread-free"
nature of the ReadProcessMemory() API).
Please note that I'm *absolutely* not certain that this is what *actually*
happens actually. I wish a *real* insider commented this...
Tomas Restrepo
> > Do you know the reason for that ? Why .tls gives problems while .data
> works
> > pretty fine ?
>
> different threads get different views of the same VM area. Much like to how
> separate processes have private address spaces. It may be, therefore, that
> access to this area makes sense only in context of a particular thread...
> obviously, this condition does not hold for a debugger (OS can't map the
> "virtualized" address to a physical page because it can't identify the
> thread for which the address is being virtualized, due to the "thread-free"
> nature of the ReadProcessMemory() API).
Consider the following:
A thread accesses tls data through it's TLS array. A pointer to this array is
stored at offset 0x2C (for x86 systems) of the TEB (thread environment block).
Since the TEB is stored in user address space, then so is TLS data. The problems
is that space for TLS slots is allocated by NT in the process default heap,
which means that, in turn, different threads will, in fact, read data from
different memory addresses for the same variable. All this is transparent to the
threads because they don't access data in TLS like normal variables. Instead,
the data is accessed by taking the tls array pointer in the TEB and adding to it
the tls index for the variable multiplied by sizeof(DWORD).
Since data in the .tls section of a PE is really stored in TLS storage, I
believe it would not be valid to actually read that addresses directly (as it's
used by the loader to create the TLS directory). In fact, down inside, the
program will have code to retrieve the TLS index from the loader when it loads.
Hope any of this helps...
--
Tomas Restrepo
tom...@mvps.org
http://members.xoom.com/trestrep/
Gary Chanson
Tomas Restrepo <tom...@mvps.org> wrote in message
news:#$ZOWul8#GA.196@cppssbbsa03...
>
> A thread accesses tls data through it's TLS array. A pointer to this array
is
> stored at offset 0x2C (for x86 systems) of the TEB (thread environment
block).
> Since the TEB is stored in user address space, then so is TLS data. The
problems
> is that space for TLS slots is allocated by NT in the process default
heap,
> which means that, in turn, different threads will, in fact, read data from
> different memory addresses for the same variable. All this is transparent
to the
> threads because they don't access data in TLS like normal variables.
Instead,
> the data is accessed by taking the tls array pointer in the TEB and adding
to it
> the tls index for the variable multiplied by sizeof(DWORD).
When I retrieve this pointer in NT, I'm getting a zero. I also get zero
for this pointer in the structure passed by the debugging API in the
CreateProceess event. Both of these pointers are usable in Win95. The
program is a Forth development system which does not define a TLS section
but does use TLS variables directly. NT is obviously allocating a TLS array
(because it works!) but my debugger can't seem to retrieve a pointer to it!
--
-GJC
-gcha...@shore.net
Slava M. Usov
Tomas Restrepo <tom...@mvps.org> wrote in message
...
> Consider the following:
>
> A thread accesses tls data through it's TLS array. A pointer to this array
is
> stored at offset 0x2C (for x86 systems) of the TEB (thread environment
block).
> Since the TEB is stored in user address space, then so is TLS data. The
problems
> is that space for TLS slots is allocated by NT in the process default
heap,
> which means that, in turn, different threads will, in fact, read data from
> different memory addresses for the same variable. All this is transparent
to the
> threads because they don't access data in TLS like normal variables.
Instead,
> the data is accessed by taking the tls array pointer in the TEB and adding
to it
> the tls index for the variable multiplied by sizeof(DWORD).
Tomas,
thanks for a nice intro to TLS :-), but what you're talking about is
slightly different from what I'm concerned of. I believe what you have
described is how the __declspec(thread) attribute works, but this makes no
sense for the current thread (of the forum). I'll explain:
when you have something like
void foo()
{
__declspec(thread) int i;
// ...
i++;
// ...
}
"i++" actually compiles to
teb->tls_array[I_INDEX]++;
or something conceptually congruent to that. But, eventually, this will
reference the VM, and this is when Marco's code gets executed. The debugger
only wakes up when access to real VM (real virtual memory... I just like it)
is being attempted. This is why it's completely irrelevant how the compiler
supports __declspec(thread).
But something strange happens: when an innocent instruction like
MOV EAX, [an_address_in_the_tls_section]
is trapped, an_address_in_the_tls_segemnt can't be read by the debugger but
appears to work great in the debuggee. This is the problem - the debugee can
read (and does read!) an address in one "special" section, but the debugger
can't. What's so special about .tls?
> Since data in the .tls section of a PE is really stored in TLS storage, I
> believe it would not be valid to actually read that addresses directly (as
it's
> used by the loader to create the TLS directory). In fact, down inside, the
> program will have code to retrieve the TLS index from the loader when it
loads.
This still does not rectify what kind of hell is kept in the .tls section
and how it's used. Any pointers?
Tomas Restrepo
>
> thanks for a nice intro to TLS :-), but what you're talking about is
> slightly different from what I'm concerned of. I believe what you have
> described is how the __declspec(thread) attribute works, but this makes no
> sense for the current thread (of the forum). I'll explain:
>
> when you have something like
>
> void foo()
> {
> __declspec(thread) int i;
>
> // ...
>
> i++;
>
> // ...
> }
>
> "i++" actually compiles to
>
> teb->tls_array[I_INDEX]++;
>
> or something conceptually congruent to that. But, eventually, this will
> reference the VM, and this is when Marco's code gets executed. The debugger
> only wakes up when access to real VM (real virtual memory... I just like it)
> is being attempted. This is why it's completely irrelevant how the compiler
> supports __declspec(thread).
Not really. I understood the question, but to answer the question you have to
understand how TLS works, specially, from the compilers point of view.
>
> But something strange happens: when an innocent instruction like
>
> MOV EAX, [an_address_in_the_tls_section]
>
> is trapped, an_address_in_the_tls_segemnt can't be read by the debugger but
> appears to work great in the debuggee. This is the problem - the debugee can
> read (and does read!) an address in one "special" section, but the debugger
> can't. What's so special about .tls?
That was precisely my point. The debugger can't read it because it's not really
there! The .tls section is not really "mapped" like the rest of the file, in the
sense that the image loader actively looks for it, and creates the tls directory
from it. Here's a quote from the PE file spec regarding .tls:
"The Microsoft run-time library facilitates this process by defining a memory
image of the TLS Directory and giving it the special name "__tls_used" (Intel
x86 platforms) or "_tls_used" (other platforms). The linker looks for this
memory image and uses the data there to create the TLS Directory. Other
compilers that support TLS and work with the Microsoft linker must use this same
technique."
...
"The code uses the TLS index and the TLS array location (multiplying the index
by four and using it as an offset to the array) to get the address of the TLS
data area for the given program and module. Each thread has its own TLS data
area, but this is transparent to the program, which doesn't need to know how
data is allocated for individual threads."
In other words, once the PE file is loaded, the .tls section doesn't hold data
per se. However, I've just written a small program that simply reads it's own
.tls section, and it can read it fine (using ReadProcessMemory), and the .tls
section seems to be mapped there just fine. I was even able to make the program
start another copy of itself, with himself as the debugger, and once again, a
call to ReadProcessMemory() was able to read the memory just fine. Also, the
.tls section is usually marked as PAGE_READWRITE or PAGE_WRITECOPY (this one is
cutios, though: dumpbin says the section is marked as PAGE_READWRITE, while a
VirtualProtect call will report PAGE_WRITECOPY).
Anyway, as I explained (and you correctly guessed before), the thread in the
debuggee can access the data, because it's really reading memory from
another spot in the address space. This makes perfect sense, because it would be
imposible for the compiler to predict how many threads would be created by the
process and reserve enough space in the .tls section for all of them.
Instead, the .tls section simply holds initialization data and the TLS index
(well, it can also contain pointers to callback routines, but afaik, they are
not currently used). Only the thread can access the data, becasue only it knows
(through the TLS array pointer in the TEB) where the data is really located. The
debugger doesn't know anything about it, unless it actually does the same thing
the thread does to access it.
>
> > Since data in the .tls section of a PE is really stored in TLS storage, I
> > believe it would not be valid to actually read that addresses directly (as
> it's
> > used by the loader to create the TLS directory). In fact, down inside, the
> > program will have code to retrieve the TLS index from the loader when it
> loads.
>
> This still does not rectify what kind of hell is kept in the .tls section
> and how it's used. Any pointers?
See above. In any case, read the .tls docs in the PE specification. It's under
Special Sections.
There's nothing much to it. For example, the sample program I wrote keeps this
in tls:
__declspec(thread) int a = 1, b = 2, c = 3;
__declspec(thread) DWORD dwAddress = 0x0042B000;
The hex dump of the .tls section from memory while the process is running is:
0042B000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B0A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B0B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B0C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B0D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B0E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B0F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B100 00 00 00 00 01 00 00 00 02 00 00 00 03 00 00 00 ................
0042B110 00 B0 42 00 00 00 00 00 00 00 00 00 00 00 00 00 .冰.............
0042B120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0042B140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
Most interesting are address 0042B100 through 0042B120, where you can clearly
see the initial values of the 4 variables.
Hope this helps
Slava M. Usov
...
> Anyway, as I explained (and you correctly guessed before), the thread in
the
> debuggee can access the data, because it's really reading memory from
> another spot in the address space.
Tomas,
you keep ignoring one little issue:
the code of debuggee contains instructions like
MOV EAX, [address_in_tls_section]
This is the instruction that is trapped, and this instruction IS REALLY
READING MEMORY FROM THIS VERY SPOT IN THE ADDRESS SPACE. This instruction
succeeds, but reading at address_in_tls_section via ReadProcessMemory()
fails.
This instruction CANNOT read anything but address_in_tls_section, UNLESS
this instruction results in an access violation, and the kernel or the CSRSS
fixes the violation silently -- but it's hardly so because this would result
in dog slow performance.
And, actually, there is a perfect reason for data in .tls being read. As
follows from the PE spec (thanks, BTW, for pointing me to it - I didn't know
static TLS stuff was described there), when a new thread is created, the
entire section is copied to some other location in the VM (which is read
later by the thread, the point you are correctly making). During this copy
the tls section IS read.
Thank you anyway for discussing it, now I think I understand how the static
TLS works. And now that I understand it, the failure to read the contents of
the TLS section from the debugger looks weird. The section must be readable!
Indeed, I launched the winword and attached a debugger to it. Then I went
through the range of addresses the .tls section is mapped to (for the
version of Word I'm using, Word 98, they are 0x3050A000 - 0x3051E000), and
they were all readable.
In a previous post, I asked Marco if that could be an alignment issue. Let's
wait for a reply from him...
P.S. The acid stuff about "virtualized" VM should be ignored! :-)
Jan Beulich
>...
>
> you keep ignoring one little issue:
>
> the code of debuggee contains instructions like
>
> MOV EAX, [address_in_tls_section]
>
I doubt this is really the case - I know of no compiler generating this
kind of code for .tls accesses. Instead, like Tomasz pointed out, all
these accesses incur an additional level of indirection.
>...
--
-----------------------------
Jan Beulich
Novell Developer Support
email: DevSu...@Novell.com
Please, respond to news group messages only by another posting, not by
email.
Slava M. Usov
Jan Beulich <jbeu...@novell.com> wrote in message
news:37CBB4...@novell.com...
> Slava M. Usov wrote:
> >...
> >
> > you keep ignoring one little issue:
> >
> > the code of debuggee contains instructions like
> >
> > MOV EAX, [address_in_tls_section]
> >
>
> I doubt this is really the case - I know of no compiler generating this
> kind of code for .tls accesses. Instead, like Tomasz pointed out, all
> these accesses incur an additional level of indirection.
You're missing the point. No thread ever accesses the tls section after the
contents of the memory has been copied to a thread-specific memory location.
During this initialization, however, the thread (or another thread) does
read the tls section, probably as
mov ecx, SIZE_OF_TLS_RAW_DATA
mov esi, START_OF_THE_RAW_DATA_TLS_SECTION
mov EDI, START_OF_THE_RAW_DATA_THREAD_SPECIFIC
rep movsd
DWORD in the range START_OF_THE_RAW_DATA_TLS_SECTION to
START_OF_THE_RAW_DATA_TLS_SECTION + SIZE_OF_TLS_RAW_DATA are in the .tls
section.
The instruction I used (MOV EAX, [address_in_tls_section]) was only a
placeholder for "some instruction that accesses memory in the .tls section".
It does not matter what these instructions really are, all that matters is
they do *access* the .tls section.
Tomas Restrepo
> you keep ignoring one little issue:
>
> the code of debuggee contains instructions like
>
> MOV EAX, [address_in_tls_section]
>
> READING MEMORY FROM THIS VERY SPOT IN THE ADDRESS SPACE. This instruction
> succeeds, but reading at address_in_tls_section via ReadProcessMemory()
> fails.
I'm not ignoring it, I just don't know where it's coming from. A simple read
shouldn't cause any problem. A write, on the other hand, would imply that the
page get's copied the first time, as aparently is PAGE_WRITECOPY protected.
In any case, both you and I have verified that the section is readable, so the
problem seems to be something else.
>
> In a previous post, I asked Marco if that could be an alignment issue. Let's
> wait for a reply from him...
OK, let's see...
>
> P.S. The acid stuff about "virtualized" VM should be ignored! :-)
Fair enough :)
Gary Chanson
Slava M. Usov <stripit...@usa.net> wrote in message
news:uA4#pH78#GA.230@cppssbbsa05...
>
> You're missing the point. No thread ever accesses the tls section after
the
> contents of the memory has been copied to a thread-specific memory
location.
> During this initialization, however, the thread (or another thread) does
> read the tls section, probably as
>
> mov ecx, SIZE_OF_TLS_RAW_DATA
> mov esi, START_OF_THE_RAW_DATA_TLS_SECTION
> mov EDI, START_OF_THE_RAW_DATA_THREAD_SPECIFIC
> rep movsd
>
> DWORD in the range START_OF_THE_RAW_DATA_TLS_SECTION to
> START_OF_THE_RAW_DATA_TLS_SECTION + SIZE_OF_TLS_RAW_DATA are in the .tls
> section.
I don't know what VC++ compiles to access TLS data, but when I access
it, I use the TLSSetValue and TLSGetValue functions. I expect that the
compiler does something similar.
--
-GJC
-gcha...@shore.net
Gary Chanson
Gary Chanson <gcha...@no.spam.shore.net> wrote in message
news:2DRy3.725$1k5.2...@news.shore.net...
>
> I don't know what VC++ compiles to access TLS data, but when I access
> it, I use the TLSSetValue and TLSGetValue functions. I expect that the
> compiler does something similar.
I took a look at what VC++ does and it's not quite what I expected.
I compiled a trivial program using a couple of thread variables:
__declspec(thread) int i = 0x12345678;
__declspec(thread) int j = 0x9abcdef0;
int main ()
{
return i + j;
}
The compiler used a single TLS slot to store a pointer to a data table
for each thread. When a thread variable is accessed, it fetches the
pointer, but not using TLSGetValue. Instead, it uses:
0040A433 mov eax,[__tls_index(0x00413e80)]
0040A438 mov ecx,dword ptr fs:[2Ch]
0040A43F mov edx,dword ptr [ecx+eax*4]
where 0x00413e80 is a global variable holding the TLS index of the pointer.
I assume that this is equivalent to TLSGetValue, but much faster.
Interestingly, the compiler wasn't smart enough to use this pointer twice.
Instead it followed with:
0040A442 mov eax,[__tls_index(0x00413e80)]
0040A447 mov ecx,dword ptr fs:[2Ch]
0040A44E mov eax,dword ptr [ecx+eax*4]
even though edx already has the same pointer. Also, it seems to allocate a
chunk of unused space in the data table:
0040A451 mov ecx,dword ptr [edx+104h]
0040A457 add ecx,dword ptr [eax+108h]
0040A45D mov eax,ecx
--
-GJC
-gcha...@shore.net
Marco Sambin
problems.
Here are some answers (and questions, obviously... ;-) ).
When ReadProcessMemory fails, if I go watching for the current page protection
(I'm talking about addresses belonging to the .tls section), I obtain
PAGE_WRITECOPY.
The VirtualProtectEx function fails (yes, the access is DWORD aligned) if I
specify (mbi.Protect | PAGE_READWRITE) as new protection (mbi.Protect is the
previous page protection, obtained with VirtualQueryEx). On the contrary, it
succeeds if I simply specify PAGE_READWRITE as the new page protection. Is this
because PAGE_WRITECOPY and PAGE_READWRITE are incompatible ? Changing in this
way the page protection could cause problems ? Is it normal that a
PAGE_WRITECOPY protection causes ReadProcessMemory's failure ?
Another question: I think I'm a little bit confused about TLS. What I
understood about TLS is that whenever a process switches threads,a new set of
physical memory pages is mapped to the .tls section's address space.
Is this not correct ? Are you saying that the *actual* thread-local data are
located (or copied) at addresses not belonging to the .tls range of addresses
?? Could you clarify a little bit this ? Why the .tls section's pages are
marked as WRITECOPY ?
Thanks a lot,
Marco.
"Slava M. Usov" wrote:
> Tomas Restrepo <tom...@mvps.org> wrote in message
> news:#qy8Mtz8#GA....@cppssbbsa02.microsoft.com...
>
> ...
>
> > Anyway, as I explained (and you correctly guessed before), the thread in
> the
> > debuggee can access the data, because it's really reading memory from
> > another spot in the address space.
>
> Tomas,
>
> you keep ignoring one little issue:
>
> the code of debuggee contains instructions like
>
> MOV EAX, [address_in_tls_section]
>
> This is the instruction that is trapped, and this instruction IS REALLY
> READING MEMORY FROM THIS VERY SPOT IN THE ADDRESS SPACE. This instruction
> succeeds, but reading at address_in_tls_section via ReadProcessMemory()
> fails.
>
> This instruction CANNOT read anything but address_in_tls_section, UNLESS
> this instruction results in an access violation, and the kernel or the CSRSS
> fixes the violation silently -- but it's hardly so because this would result
> in dog slow performance.
>
> And, actually, there is a perfect reason for data in .tls being read. As
> follows from the PE spec (thanks, BTW, for pointing me to it - I didn't know
> static TLS stuff was described there), when a new thread is created, the
> entire section is copied to some other location in the VM (which is read
> later by the thread, the point you are correctly making). During this copy
> the tls section IS read.
>
> Thank you anyway for discussing it, now I think I understand how the static
> TLS works. And now that I understand it, the failure to read the contents of
> the TLS section from the debugger looks weird. The section must be readable!
>
> Indeed, I launched the winword and attached a debugger to it. Then I went
> through the range of addresses the .tls section is mapped to (for the
> version of Word I'm using, Word 98, they are 0x3050A000 - 0x3051E000), and
> they were all readable.
>
> In a previous post, I asked Marco if that could be an alignment issue. Let's
> wait for a reply from him...
>
> P.S. The acid stuff about "virtualized" VM should be ignored! :-)
>
Slava M. Usov
Marco Sambin <sam...@tin.it> wrote in message
news:37CD3147...@tin.it...
> Sorry for the short "absence", but I was trying to understand something
with my
> problems.
>
> Here are some answers (and questions, obviously... ;-) ).
> When ReadProcessMemory fails, if I go watching for the current page
I still don't understand why ReadProcessMemory() fails on a page from the
.tls section. I tried reading this section, it worked great. Are you sure it
actually was the .tls section?
> protection
> (I'm talking about addresses belonging to the .tls section), I obtain
> PAGE_WRITECOPY.
> The VirtualProtectEx function fails (yes, the access is DWORD aligned) if
I
> specify (mbi.Protect | PAGE_READWRITE) as new protection (mbi.Protect is
the
> previous page protection, obtained with VirtualQueryEx). On the contrary,
Calling VPE() with PAGE_READWRITE | PAGE_WRITECOPY is invalid. Only *one* of
them may be specified.
> it
> succeeds if I simply specify PAGE_READWRITE as the new page protection. Is
this
> because PAGE_WRITECOPY and PAGE_READWRITE are incompatible ? Changing in
this
> way the page protection could cause problems ?
As I wrote above, it's a kind of "syntax error". You might have been fooled
by doing VPE(mbi.Protect | PAGE_GUARD), but PAGE_GUARD and PAGE_NOCACHE are
*exceptions* from the rule, and it's clearly documented.
> Is it normal that a
> PAGE_WRITECOPY protection causes ReadProcessMemory's failure ?
No, this is not normal. PAGE_WRITECOPY allows the page to be read.
>
> Another question: I think I'm a little bit confused about TLS. What I
I was, too :-) Thanks Tomas I was able to figure that out.
> understood about TLS is that whenever a process switches threads,a new set
of
> physical memory pages is mapped to the .tls section's address space.
> Is this not correct ? Are you saying that the *actual* thread-local data
are
> located (or copied) at addresses not belonging to the .tls range of
addresses
This is not correct. The .tls section contains "initialization" data. When a
new thread is created, a range of memory on the system heap is allocated,
and the contents of the .tls section is copied to that range of memory.
"That range" of memory then becomes what the newly created thread considers
its view of the TLS. The thread then only accesses that range (on heap),
never the .tls section.
The only cases when the .tls does get accessed are either during an early
phase of process creation, or during creation of a new thread. The access
should be read-only.
This is how *I* understand it anyway...
> ?? Could you clarify a little bit this ? Why the .tls section's pages are
> marked as WRITECOPY ?
Because it's mapped directly to the image. Everything that is mapped to an
image (of a process or a DLL) is by default PAGE_WRITECOPY (or
PAGE_EXECUTE_WRITECOPY) or PAGE_READ (or PAGE_EXECUTE_READ), never
PAGE_READWRITE (or PAGE_EXECUTE_READWRITE). This is to prevent an errant (or
malice) process from modifying the image and pages in other processes. When
you do write to a region protected with PAGE_WRITECOPY (or change its
protection with VPX()), the page is substituted with another page (backed by
pagefile) and its protection changes to PAGE_READWRITE.
.tls should not be normally written to (well, perhaps loader might do that,
but you won't see that anyway) because it stores only initialization data.
jw...@oro.net
In release it is.
Sent via Deja.com http://www.deja.com/
Share what you know. Learn what you don't.
Tomas Restrepo
> of
> > physical memory pages is mapped to the .tls section's address space.
> > Is this not correct ? Are you saying that the *actual* thread-local data
> are
> > located (or copied) at addresses not belonging to the .tls range of
> addresses
But that's precisely what happens. As Slava already pointed out, the .tls
section is only used by the loader to initialize the TLS directory and find the
initial values of all variables stored in TLS. Technically speaking, the system
*could* implement tls the way you describe, but it's not a good idea. Why?
because:
1- It would require the memory manager to keep track of memory pages and memory
address spaces on a per-thread basis, instead of a per-process basis, which
would be much more expensive.
2- Remember that the .tls section is really a loader extension, and it's not the
only way to implement TLS. In fact, it's not even the preferred way to implement
TLS because a dll that uses __declspec(thread) cannot be loaded dynamically
using LoadLibrary(). That's why using the Tls*() apis is the preferred method.
Marco Sambin
TlsAlloc and then set with TlsSetValue() end up ?
Thanks again to everybody for the complete explanations.
Marco Sambin.
Tomas Restrepo
> What's different when using the Tls*() APIs ? Where do the variables allocated
with
> TlsAlloc and then set with TlsSetValue() end up ?
They end up in the same place, the heap. The difference is that the bulk of the
work of setting up TLS for the threads is performed by the code itself (via
TlsAlloc()), instead of by the image loader.
The image loader, on the other hand, cannot initialize TLS properly for an image
with a .tls section if the image is loaded diynamically using LoadLibrary().
This is because the image itself is loaded after threads are already running in
the process context, which prevents the loader from properly initializing the
TLS directory.
Marco Sambin
different runs of the same program and I noticed something strange (to me):
even for two identical executions (same program, same inputs) , the .data
sections relative to each execution are different at the end of the program. I
must specify that the program is a VERY simple one, performing some arithmetic
computations and then exiting (no GUI).
What can these "variable" data be ? Are they located in particular regions of
the .data section ?
Thanks a lot,
Marco.
"Slava M. Usov" wrote:
> > understood about TLS is that whenever a process switches threads,a new set
> of
> > physical memory pages is mapped to the .tls section's address space.
> > Is this not correct ? Are you saying that the *actual* thread-local data
> are
> > located (or copied) at addresses not belonging to the .tls range of
> addresses
>
Slava M. Usov
> These last days I was trying to compare the .data sections at the end of
> different runs of the same program and I noticed something strange (to
me):
> even for two identical executions (same program, same inputs) , the .data
> sections relative to each execution are different at the end of the
program. I
> must specify that the program is a VERY simple one, performing some
arithmetic
> computations and then exiting (no GUI).
> What can these "variable" data be ? Are they located in particular regions
of
> the .data section ?
Marco,
I don't understand what you're talking about. Do you mean that the .data
section gets loaded at different offsets from the image base address? This
is simply not possible - the PE loader won't do it. The base image address,
however, *may* change (thus the .data section will be at another address)
but it does not normally change between runs on the *same* system. For, as
you said, a very simple program it should not ever change at all (because
there is nothing that prevents the image from being loaded at the predefined
address).
Please clarify.
BTW, what is the status of ReadProcessMemory() problem?
Marco Sambin
sorry for having been not so clear...
To say the truth, it's a friend of mine (who is working on similar things as
me) that is doing what I tried to describe...
I try to be clearer: he's trying to compare the CONTENTS of the .data section
after identical executions of the same program. Since this section should
contain the initialized global and static variables, I would say that its
contents should be the same after identical execution, because the global
variables have the same values after two identical executions.
Nevertheless, he said to me that this doesn't happen: there are regions
belonging to the .data section that have different contents after two identical
executions of the same program.
Is it clearer ? Is it possible ?
For what's concerning the ReadProcessMemory() problem, I haven't investigated
so much on it in the last few days: since after changing the page protection to
READWRITE I'm able to read, I didn't stay on it so much. I will come back to it
in the next days anyway, because it's strange I'm not able to read from
WRITECOPY protected pages... I will let you know.
Many thanks,
Marco.
Slava M. Usov
> Hi Slava,
> sorry for having been not so clear...
>
> To say the truth, it's a friend of mine (who is working on similar things
as
> me) that is doing what I tried to describe...
> I try to be clearer: he's trying to compare the CONTENTS of the .data
section
> after identical executions of the same program. Since this section should
> contain the initialized global and static variables, I would say that its
> contents should be the same after identical execution, because the global
> variables have the same values after two identical executions.
> Nevertheless, he said to me that this doesn't happen: there are regions
> belonging to the .data section that have different contents after two
identical
> executions of the same program.
> Is it clearer ? Is it possible ?
Quite possible. Run the dumpbin utility against the program in question -
chances are 99% that the .data protection will be READ WRITE. This is the
default protection that Microsoft tools put on this section. As you
correctly notice the section holds global variables, and it's perfectly OK
for the program to change them.
Why the program behaves "non-deterministically" can not be said
deterministically :-) Well, I bet even the very simple arithmetic program
without GUI does link to the CRT library.... right? The CRT library really
goes through all sorts of gyrations when the program is started. It may
store, for instance, time, and other kinds of volatile contents into the
global variables. Try linking the program without the CRT...
Marco Sambin
is what happens:
With more complex applications (SOME multithreaded apps) certain memory
locations belonging to the .data section change without my monitor detects
write accesses to those locations. I make a copy of the .data section when I
receive the very first breakpoint and then, at certain point, I have some
memory locations modified apparently without any write access to those
addresses. I looks like I'm missing some accesses. Following your suggestions,
I suspend all the threads (except the one raising the exception) when I
receive the GUARD_PAGE exception and I resume them when I receive the
immediately following SINGLE_STEP exception in my own debugger (before calling
the ContinueDebugEvent() function).
What could be the reason of that behavior ??
Thanks a lot,
Marco.
Slava M. Usov
If you're absolutely sure there are no other running threads in the
debuggee... BTW, when do you suspend them? Before reading the memory
location or after that? Well, assuming all the threads are suspended
*before*, the only thing that may access the memory is either threads of
some other process, or the system itself.
If the memory is shared between two (or more) processes, and it's not
write-copied on write, the guard attribute of the page should also be shared
between the processes... and one process may hit the page and reset the
attribute without raising the exception in the other one... hmm... needless
to say, it's only my 2 euros speculations, and you should test if it's
really so (if you don't, I will, but not today and not tomorrow). If there
are no cooperating processes, this is definitely not the case.
[Sections mapped to an executable or a DLL are always write-copied unless
there is an explicit flag set by the linker... it's rarely so; the only way
to share non-write-copied memory is by CreateFileMapping().]
The other option is the OS... If the application starts a ReadFile() or a
similar operation when the guard attribute is not set, the driver locks the
buffer and it will be written then regardless of your attempts to set the
guard attribute on it (the attempts may be failed *or* ignored by the OS,
BTW... this is where I'm not certain, although I'd vote for failing).
Marco Sambin
> If you're absolutely sure there are no other running threads in the
> debuggee...
I'm pretty sure: I insert information relative to the thread in a list each
time my debugger receives a CREATE_THREAD debug event. Then, when I receive the
GUARD_PAGE exception, I suspend all the threads in that list except the one
raising the exception, and the SuspendThread calls are all successful... Is it
enough to be sure or should I try something else ?
> BTW, when do you suspend them? Before reading the memory
> location or after that?
I suspend the threads BEFORE reading from memory and I resume them AFTER having
restored the GUARD attribute.
> Well, assuming all the threads are suspended
> *before*, the only thing that may access the memory is either threads of
> some other process, or the system itself.
>
> If the memory is shared between two (or more) processes, and it's not
> write-copied on write, the guard attribute of the page should also be shared
> between the processes... and one process may hit the page and reset the
> attribute without raising the exception in the other one... hmm... needless
> to say, it's only my 2 euros speculations, and you should test if it's
> really so (if you don't, I will, but not today and not tomorrow). If there
> are no cooperating processes, this is definitely not the case.
I think there are no cooperating processes, at least apparently. In fact, one
of the applications that causes this kind of problems is MSExcel: I don't think
this application shares its memory with any other process, don't you ?
BTW, with MSWord (that should be quite similar to Excel), everything works
fine.
Thank you,
Marco Sambin.
J. Wesley Cleveland
Marco Sambin wrote:
[snip]
>
> I think there are no cooperating processes, at least apparently. In fact, one
> of the applications that causes this kind of problems is MSExcel:
One odd thing I discovered about excel is that it has a keyboard hook
which does something significant. I don't know which thread contexts
hooks operate in.
Slava M. Usov
J. Wesley Cleveland <jw...@oro.net> wrote in message
news:37E65539...@oro.net...
Hooks (global hooks) operate in the context of the processes and threads
that happen to be processing a particular type of messages right now. The
(global) hooks accomplish it by injecting a DLL into *all* processes. The
hooks are normally planted to communicate some data back to the parent
process (and do other nasty things), and they may use shared memory...
If MSExcel does use a global hook, we do have cooperating processes, and
this implies...
Marco Sambin
> Hooks (global hooks) operate in the context of the processes and threads
> that happen to be processing a particular type of messages right now. The
> (global) hooks accomplish it by injecting a DLL into *all* processes. The
> hooks are normally planted to communicate some data back to the parent
> process (and do other nasty things), and they may use shared memory...
What do you mean with parent process ? What could be the parent process in
the case of Excel ?
If this was the case, would there be a way not to miss "external" accesses ?
The idea that the problems I'm having are not strictly related to
multithreading and suspend/resume issues is getting stronger in my mind:
I've tried to make a simple multithreaded application where the threads are
accessing the same data, and everything is working fine with my memory
access tracker. All the read/write accesses are correctly detected...
I've also thought that Excel could call VirtualProtect and eliminate the
GUARD attribute on the .data pages: but this should not be the case, because
I tested Excel with my API spying utility and no trace of significant
VirtualProtect s appears.
Thank you for the help,
Marco.
Tomas Restrepo
>Hooks (global hooks) operate in the context of the processes and threads
> that happen to be processing a particular type of messages right now. The
> (global) hooks accomplish it by injecting a DLL into *all* processes. The
> hooks are normally planted to communicate some data back to the parent
> process (and do other nasty things), and they may use shared memory...
>
> If MSExcel does use a global hook, we do have cooperating processes, and
> this implies...
I would doubt excel sets up a global hook. It's much more likely that it just
installs a local hook, which is quite common (MFC does it all the time)....
It's just a guess, though
Slava M. Usov
> "Slava M. Usov" wrote:
>
> > Hooks (global hooks) operate in the context of the processes and threads
> > that happen to be processing a particular type of messages right now.
The
> > (global) hooks accomplish it by injecting a DLL into *all* processes.
The
> > hooks are normally planted to communicate some data back to the parent
> > process (and do other nasty things), and they may use shared memory...
>
> the case of Excel ?
The "parent" process is the process that installs global hooks (parent of
hooks, that is).
> If this was the case, would there be a way not to miss "external" accesses
?
It gets trickier and trickier... as the processes that have been "hooked-in"
run outside your debugger, you can't do much about them... they may open
shared sections but you *cannot* (at least 100% reliable) tell if a view of
a particular section is equivalent to a view of a section in another
process... NT kernel (executive) can do it, naturally, but you really need
help of a kernel insider, and probably you will have to write a KM driver to
achieve that (to identify a section object by an address range in the
process being debugged, to watch for other processes opening the section
object, and to determine the address range it's being mapped into. Or some
other way around it, I'm not a KM expert at all.)
Slava M. Usov
> Slava,
>
> >Hooks (global hooks) operate in the context of the processes and threads
> > that happen to be processing a particular type of messages right now.
The
> > (global) hooks accomplish it by injecting a DLL into *all* processes.
The
> > hooks are normally planted to communicate some data back to the parent
> > process (and do other nasty things), and they may use shared memory...
> >
> > this implies...
>
> I would doubt excel sets up a global hook. It's much more likely that it
just
> installs a local hook, which is quite common (MFC does it all the
time)....
> It's just a guess, though
Tomas,
I'm not promoting it does :-) I'm just telling it *might*.
J. Wesley Cleveland
Another question is: If a thread has installed a local keyboard hook and
the thread is suspended, will the hook still be activated ?
Slava M. Usov
If it's local, it may only work in the context of "this" thread; also, it is
strictly a user-mode thing. When a thread is suspended, it cannot do
anything in the user mode. So I vote for "a local hook cannot be activated
when its thread is suspended". I may be wrong, though. This question must
probably be asked in a GUI group...
Tomas Restrepo
> the thread is suspended, will the hook still be activated ?
Well, I haven't verified it (and don't have the time to do it now), but the docs
on WH_KEYBOARD quite clearly state:
"The KeyboardProc hook procedure is an application-defined or library-defined
callback function used with the SetWindowsHookEx function. The system calls this
function whenever an application calls theGetMessage orPeekMessage function and
there is a keyboard message (WM_KEYUP orWM_KEYDOWN) to be processed. "
So, if the thread _has_ to call GetMessage() or PeekMessage() for the hook
function to get invoked, I think it's fairly safe to assume that if the thread
is suspended the hook won't be activated :)
Marco Sambin
global hook, can it share the region of memory containing the exe's global
variables ? All the regions of memory can be shared ?
BTW, Excel's case is not so isolated, since even with Autocad 14 I get
something similar...
Thank you,
Marco.
"Slava M. Usov" wrote:
> > Another question is: If a thread has installed a local keyboard hook and
> > the thread is suspended, will the hook still be activated ?
>
Marco Sambin
- Excel installs two hooks, one of type WH_KEYBOARD, one of type WH_MSGFILTER
(= two calls to SetWindowsHookEx() are logged)
- Both the hooks, in the particular execution I made, were associated with the
thread having 0x92 as TID
- By analyzing Excel with my Debug events logger, I noticed that the thread
with ID 0x92 happens to be Excel's main thread.
This seems to demonstrate that the hook is *local* and this shouldn't create
problems to my memory monitor.
But it does.
Comments ?
Slava M. Usov
> I've just analyzed Excel with my API spying utility and the results are:
> - Excel installs two hooks, one of type WH_KEYBOARD, one of type
WH_MSGFILTER
> (= two calls to SetWindowsHookEx() are logged)
> - Both the hooks, in the particular execution I made, were associated with
the
> thread having 0x92 as TID
> - By analyzing Excel with my Debug events logger, I noticed that the
thread
> with ID 0x92 happens to be Excel's main thread.
>
> This seems to demonstrate that the hook is *local* and this shouldn't
create
> problems to my memory monitor.
> But it does.
>
> Comments ?
How about the good old ReadFile() or something? Or, are there any
VirtualProtect() or VirtualProtectEx() calls?
Marco Sambin
> How about the good old ReadFile() or something? Or, are there any
> VirtualProtect() or VirtualProtectEx() calls?
There is no trace of ReadFile, at least up to the moment my problems show up.
Moreover, making a real-time analysis, I discovered that the hooks (that are
anyway local) are installed *after* my problems show up.
Finally, no VirtualProtect or VirtualProtectEx on the addresses I'm interested
in is called.
The only API calls involving the range of addresses belonging to the .data
section seem to be completely harmless and are:
-GetModuleFileName
-RegOpenKeyExW
-RegQueryValueEx
-CreateWindowExW
-GetTextFace
Any ideas ?
Thanks a lot for your support,
Marco.
J. Wesley Cleveland
Assuming it is the same addresses being changed, load excel into your
debugger, check to see if the values were changed during the loading
process, set a breakpoint on memory write, and go.
Marco Sambin
> Assuming it is the same addresses being changed, load excel into your
> debugger, check to see if the values were changed during the loading
> process, set a breakpoint on memory write, and go.
Wesley,
I'm not sure I understand what you said. The problem is that I can't detect *any*
write access to the location that gets changed (yes, the problem is always on the
same locations). What do you mean with the "loading process" ?
If you generally mean the period of time between the launching of Excel and the
moment in which Excel accepts the user input, then the answer is: yes, the value of
those memory locations changes during the loading process.
Instead, if you mean the period of time between the process creation and the moment
when my debugger receives the very first breakpoint, the answer is: no, the memory
locations change *after* the first breakpoint.
In fact, I start detecting memory accesses to the .data section only after the very
first breakpoint. I detect two read accesses to the location 0x3053E830 and a given
value is read, then I detect a third read access to that location and the read
value is now changed. The problem is that no write access to that location or to
adjacient locations is detected between the second and the third read access.
Further ideas ?
Thanks a lot,
Marco.
Slava M. Usov
> "Slava M. Usov" wrote:
>
> > How about the good old ReadFile() or something? Or, are there any
> > VirtualProtect() or VirtualProtectEx() calls?
>
> There is no trace of ReadFile, at least up to the moment my problems show
up.
> Moreover, making a real-time analysis, I discovered that the hooks (that
are
> anyway local) are installed *after* my problems show up.
> Finally, no VirtualProtect or VirtualProtectEx on the addresses I'm
interested
> in is called.
> The only API calls involving the range of addresses belonging to the .data
> section seem to be completely harmless and are:
>
> -GetModuleFileName
> -RegOpenKeyExW
> -RegQueryValueEx
> -CreateWindowExW
> -GetTextFace
>
Marco,
why do you think RegQueryValueEx() is "harmless"? It *does* write to the
memory passed to it; if it cannot write because the memory is protected (or
not committed) it does not raise exceptions but returns an error code. Much
the same thing as ReadFile() is.
Marco Sambin
> Marco,
>
> why do you think RegQueryValueEx() is "harmless"? It *does* write to the
> memory passed to it; if it cannot write because the memory is protected (or
> not committed) it does not raise exceptions but returns an error code. Much
> the same thing as ReadFile() is.
When you say that it does not raise exceptions, you mean that my debugger
wouldn't catch that access (= no PAGE_GUARD exception) ?
But I would say that if the memory is protected and it returns an error code,
it also doesn't write to memory. Am I wrong ?
Thanks a lot,
Marco.
Slava M. Usov
One *single* RegQueryValueEx() will fail; the next one will be successful
because the PAGE_GUARD attribute is reset by the first attempt. The reason
why you can't see exceptions with your debugger is because RegQueryValueEx()
is executed in the kernel mode and handles exceptions *there*... KM
exceptions are not propagated to the user mode and are invisible to a
user-mode debugger. Does your API interceptor confirm that?
Marco Sambin
> One *single* RegQueryValueEx() will fail; the next one will be successful
> because the PAGE_GUARD attribute is reset by the first attempt. The reason
> why you can't see exceptions with your debugger is because RegQueryValueEx()
> is executed in the kernel mode and handles exceptions *there*... KM
> exceptions are not propagated to the user mode and are invisible to a
> user-mode debugger. Does your API interceptor confirm that?
From my API interceptor, I notice a behavior similar to the one you described,
but for the RegOpenQueryValueEx. I know that this function is not going to
write in the memory buffer whose address is passed as parameter, but it could
anyway reset the guard attribute and then any simple memory write access would
be invisible for my monitor. Is the behavior of this API similar to the one you
described for RegQueryValueEx ?
What I see in my API log file is a call to RegOpenKeyExW having as parameter an
address very near to the one where I have the problems (it could be in the same
page). This first call fails: it returns 0x80000001 (E_NOTIMPL = Not
implemented; BTW, why *this* kind of error ?). The immediately following API
call is an identical call, but now it succeeds. This seems to confirm (once
again...) your hypothesis.
Are there ways to solve this kind of problems ? Are there a lot of API
functions having a similar behavior (= not raising PAGE_GUARD exceptions even
if they try to read/write in guard-protected pages) ? If not, I could try a
kind of cooperation between my API Interceptor and my memory monitor... This
would not be simple anyway, because they are in different processes...
Thank you very much,
Marco.
Slava M. Usov
> "Slava M. Usov" wrote:
>
> > One *single* RegQueryValueEx() will fail; the next one will be
successful
> > because the PAGE_GUARD attribute is reset by the first attempt. The
reason
> > why you can't see exceptions with your debugger is because
RegQueryValueEx()
> > is executed in the kernel mode and handles exceptions *there*... KM
> > exceptions are not propagated to the user mode and are invisible to a
> > user-mode debugger. Does your API interceptor confirm that?
>
> From my API interceptor, I notice a behavior similar to the one you
described,
> but for the RegOpenQueryValueEx.
RegOpenQueryValueEx() ?? There is no such thing (at least advapi32.dll does
not export it).
> I know that this function is not going to
> write in the memory buffer whose address is passed as parameter, but it
could
> anyway reset the guard attribute and then any simple memory write access
would
> be invisible for my monitor. Is the behavior of this API similar to the
one you
> described for RegQueryValueEx ?
Again, what API are you talking about?
> What I see in my API log file is a call to RegOpenKeyExW having as
parameter an
> address very near to the one where I have the problems (it could be in the
same
> page). This first call fails: it returns 0x80000001 (E_NOTIMPL = Not
> implemented; BTW, why *this* kind of error ?).
From winnt.h:
#define STATUS_GUARD_PAGE_VIOLATION ((DWORD )0x80000001L)
> The immediately following API
> call is an identical call, but now it succeeds. This seems to confirm
(once
> again...) your hypothesis.
>
> Are there ways to solve this kind of problems ? Are there a lot of API
> functions having a similar behavior (= not raising PAGE_GUARD exceptions
even
> if they try to read/write in guard-protected pages) ? If not, I could try
a
> kind of cooperation between my API Interceptor and my memory monitor...
Sorry, but this is a question I cannot answer. In general, API functions
that accept buffers will not raise exceptions because they're all
kernel-mode; notable exceptions from the rule are the string-manipulation
routines and heap functions - because they work in user-mode.
Now, one only has to count them :-)
Marco Sambin
As I correctly wrote in the second part of my message, the API function I was
talking about is RegOpenKeyExW (I merged the name of two functions...). What I
see in my log file is what I described in my previous message.
How can I understand if an API function is kernel-mode ? Anyway, depending on
the number, I'm not sure I'm gonna count them all ... :-)
For what's concerning the error, I said that the error was E_NOTIMPL because
the VC++ documentation says that the error values returned by that function are
defined in winerrors.h , and there 0x80000001 corresponds to E_NOTIMPL. Anyway,
there is no doubt that the correct interpretation is
STATUS_GUARD_PAGE_VIOLATION...
Slava M. Usov
> Sorry for a little bit of confusion in my previous message.
> As I correctly wrote in the second part of my message, the API function I
was
> talking about is RegOpenKeyExW (I merged the name of two functions...).
Well, RegOpenKeyEx() does write to a buffer; the buffer being the variable
that is pointed by the last argument, 4 bytes long (HKEY).
> What I see in my log file is what I described in my previous message.
> How can I understand if an API function is kernel-mode ? Anyway, depending
on
> the number, I'm not sure I'm gonna count them all ... :-)
Sorry, there appears to be a lot of such APIs, and, no, you cannot tell it's
kernel-mode or not without guessing and testing.
I think it's possible to resolve this issue by combining the memory tracker
with an API interceptor... again it may be not worth it...
Marco Sambin
driver ?
Thanks,
Marco.
J. Wesley Cleveland
Sorry. When I said "your debugger", I didn't mean *your* debugger, I
meant SoftIce or MSDEV or ...
These can use the hardware breakpoint registers to detect memory writes.
It may not help because I doubt they would catch a write from kernel
mode, if that is what is occuring.
Slava M. Usov
> Definitely, do you think the final solution would be writing a kernel-mode
> driver ?
<grin> Yes... not exactly a "driver" but something that has full control of
the machine... something similar to SoftIce. I'm afraid, though, that the
DDK documentation is not enough (and not even close!)...
Ahmed Kandeel
Use the PSAPI!EnumProcessModules then PSAPI!GetModuleBaseNameA
J. Wesley Cleveland
"Slava M. Usov" wrote:
>
> Marco Sambin <sam...@tin.it> wrote in message
> news:37EDF650...@tin.it...
>
> Marco,
>
> > To your knowledge, the so called kernel-mode debugger would be a solution?
>
> I believe it shall. The problem you have encountered is due to your
> inability to monitor access to a range of memory when kernel-mode code is
> executed. Kernel-mode debuggers do not have this problem because they
> monitor all flavors of code alike, with an added bonus of being able to use
> CPU's debugging facilities, many of which are not available for user-mode
> debuggers.
[snip]
I have had similar problems using SoftIce, though I think it is a
"feature". The problem is that the breakpoint is on a specific virtual
address, and it is hard to know if that address refers to a variable in
your process if it is not your task executing (almost always it is not).
Marco Sambin
when you told me about putting a breakpoint on memory write in debuggers such as
MSDEV one or SoftIce, did you imply that it is possible to put a breakpoint on
the specific "memory write" operation or you simply meant to put a breakpoint on
the instruction making a write access to the memory ? Or still, is it possible
(from the cited debuggers) to put something as a "data breakpoint" on a memory
location (address), blocking the program as soon as that location (containing
data) is accessed (either on read or write) ?
Thanks a lot,
Marco.
Slava M. Usov
> Wesley,
>
> when you told me about putting a breakpoint on memory write in debuggers
such as
> MSDEV one or SoftIce, did you imply that it is possible to put a
breakpoint on
> the specific "memory write" operation or you simply meant to put a
breakpoint on
> the instruction making a write access to the memory ? Or still, is it
possible
> (from the cited debuggers) to put something as a "data breakpoint" on a
memory
> location (address), blocking the program as soon as that location
(containing
> data) is accessed (either on read or write) ?
Marco,
(running a short debug session with MSDEV) MSDEV seems to enter a
single-step mode when you set a breakpoint on memory change... I can *feel*
it because it runs terribly slow... as such, it detects everything but it's
certainly not what you meant by "data breakpoint"...
Since MSDEV is the only debugger I have on the machine I'm writing this at
now, I can't verify how SoftICE handles that... I do recall that it has
a BPM command, which allows setting upto 4 breakpoints on bytes, words, and
dwords, and a BPR command which allows setting an unlimited (?) amount of
breakpoints on ranges of memory. The BPM functionality is apparently
implemented via DR0-DR3 registers of Pentium and later CPUs (hence the
limitation - 4 registers 4 breakpoints), while BPR must be implemented very
much similarly to how you did with PAGE_GUARD, except that SoftICE can
actually access page tables and page directories and mark pages as absent
and then handle the page faults without the OS ever noticing that. Still I
can't verify that it handles that 100% reliably because a page (physical)
may be aliased by more than one logical page... looking up such aliases
requires a lot of internal knowledge of the OS...
Finally, I remember NTSD (NT kernel-mode debugger) does allow setting
breakpoints on memory... my experience with it is very limited, so I can't
comment... may be worth checking.
Stefan Gustafsson
On the other hand, if I remember correctly, DR0-DR3 always use what Intel
calls "linear addresses" (Which is the same thing as a Virtual address on
NT). They do not set a breakpoint on a particular physical memory address.
This means that a breakpoint at addresses below 0x80000000 are only valid in
the process that was active when you set the breakpoint.
Softice will break if any code in the system (kernel or user) accesses the
particular linear address, but only if the correct process is active.
This means that even softice might miss the fact that a certain physical
page can be modified by more than one process.
Please correct me if I am wrong as the above is written from memory, I do
not have the SoftIce manual in front of me.
/SG
Slava M. Usov <stripit...@usa.net> wrote in message
news:#f#uN9eC$GA.225@cppssbbsa05...
Slava M. Usov
Stefan Gustafsson <s...@nospam.se> wrote in message
news:#X5ONNlC$GA.73@cppssbbsa04...
> SoftIce can definitely use DR0-DR3 to set real read/write breakpoints.
>
> On the other hand, if I remember correctly, DR0-DR3 always use what Intel
> calls "linear addresses" (Which is the same thing as a Virtual address on
> NT). They do not set a breakpoint on a particular physical memory address.
>
> This means that a breakpoint at addresses below 0x80000000 are only valid
in
> the process that was active when you set the breakpoint.
>
> Softice will break if any code in the system (kernel or user) accesses the
> particular linear address, but only if the correct process is active.
>
> This means that even softice might miss the fact that a certain physical
> page can be modified by more than one process.
>
> Please correct me if I am wrong as the above is written from memory, I do
> not have the SoftIce manual in front of me.
Looking at the SoftICE manual now, here's the description of the BPM
command:
(quote)
If you do not specify a debug register, SoftICE uses the first available
debug register starting
from DR3 and working backwards. You should not include a debug register
unless you are
debugging an application that uses debug registers itself such as a
debugging tool.
(end quote)
and below, for NT usage of BPM:
(quote)
Any breakpoint set on an address below 80000000h (2 GB) is address-context
sensitive. This
includes WIN32 and DOS V86 applications. Take care to ensure you are in the
correct
context before setting a breakpoint.
(end quote)
Regarding BPR (breakpoint on range), the manual I have says it's not
supported on Windows NT!! The following quote does however shed some light
on how it's implemented:
(quote)
Range breakpoints are always set in the page tables that are active when the
BPR command is
entered. Therefore, if range addresses are below 4MB, the range breakpoint
will be tied to the
virtual machine that is current when BPR is entered. Because of this fact,
there are some areas
in memory where range breakpoints are not supported. These include the page
tables, GDT,
IDTs, LDT, and SoftICE. If you try to set a range breakpoint or back trace
range over one of
these areas, SoftICE returns an error.
(end quote)
Does anybody have the *latest* version of the SoftICE? Is BPR supported on
NT?
I can't believe "the most powerful debugger available for advanced
Windows95/NT developers" [1] has the same problem!
---
[1] ftp://ftp.numega.com/anonymous/pub/readme_si.txt:
(quote)
Welcome to the SoftICE Evaluation Version
-----------------------------------------
Thank you for choosing to evaluate SoftICE! SoftICE is the most powerful
debugger available for advanced Windows95/NT developers.
(end quote)
Marco Sambin
PAGE_NO_ACCESS rather than PAGE_GUARD attribute, would there be differences
about the behavior of my memory monitor when global variables are passed as
parameters to API calls that gets executed in kernel-mode ? It is true that
probably even in this case the exception would be handled within the kernel and
my debugger would not be notified, but I'm not so sure about it. While the
documentation clearly says that if a system service tries to access a Guard
page the system fails but no exception is raised, nothing similar is said about
PAGE_NO_ACCESS violation.
Is it not possible that the GP fault is notified to the debugger anyway ?
Thanks a lot for any comment,
Marco.
Slava M. Usov
> A thing got to my mind a few days ago: if I protected the pages with
> PAGE_NO_ACCESS rather than PAGE_GUARD attribute, would there be
differences
> about the behavior of my memory monitor when global variables are passed
as
> parameters to API calls that gets executed in kernel-mode ? It is true
that
> probably even in this case the exception would be handled within the
kernel and
> my debugger would not be notified, but I'm not so sure about it. While the
> documentation clearly says that if a system service tries to access a
Guard
> page the system fails but no exception is raised, nothing similar is said
about
> PAGE_NO_ACCESS violation.
> Is it not possible that the GP fault is notified to the debugger anyway ?
Hardly there is a difference. A typical API, I believe, is a thing like:
DWORD API(...)
{
__try
{
// ...
}
__except( EXCEPTION_EXECUTE_HANDLER )
{
return GetExceptionCode();