Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Debugging aspnet_wp.exe dump

66 views
Skip to first unread message

Cheung, Jeffrey Jing-Yen

unread,
Jul 1, 2003, 6:47:48 PM7/1/03
to
Before I get to my question, here is some background info.

My previous experience in windbg is very minimal. I usually only use it
when I get a dump during a BSOD to see what conflicts occurred. I am a
master at !analyze -v only when symbols are correctly loaded, as it
usually points me directly to the source of the problem.

Recently, an ASP.NET application has been consistently deadlocking in a
production environment. MS states that I will have to produce a dump
file, and debug in order to pinpoint exactly where this deadlock is
taking place. I have accomplished in getting a dmp file per MS's
instructions, but I can't make heads or tails out of it. A lot of the
.NET assemblies have symbols readily available in their respective bin
folders, but they don't seem to be loaded.

My actual question is, how do I find exactly what is going rogue? From
what very little I've read, ASPNET has multiple threads, and usually in
a deadlock scenario, these threads are all waiting on each other. I'm
not certain how to go about finding this problem. Also, will the
functionality of loading the .NET assembly symbols benefit me at all
while debugging, or is it pointless? If it will be beneficial, how do I
do this?

Thanks in advance for any replies.

Jeff Cheung

Ivan Brugiolo [MSFT]

unread,
Jul 1, 2003, 10:44:19 PM7/1/03
to
Start from the output of the '~*kb' command.
Look at the top function in the stack, and identify what are the threads
waiting for.

You may find threads waiting on Window Messages
(normally the message pump of an STA thread),
you may find thread in the LPC WaitReceiveReplay state (RPC worker threads),
you may find threads doing an explicit wait on an event,
or you may find threads waiting on Critical Sections.

I would attack the critical section first,
even if I would suspect an STA/MTA deadlock
given the generic cisrcumstances.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"Cheung, Jeffrey Jing-Yen" <jin...@emailREMOVEMEuser.net> wrote in message
news:#BIbLLCQ...@TK2MSFTNGP10.phx.gbl...

Cheung, Jeffrey Jing-Yen

unread,
Jul 2, 2003, 12:20:03 PM7/2/03
to
Ivan,

First of all, I appreciate your response. I am not that familiar with
the debugging environment, and this is something that I want to rectify
over time.

From the looks of things, it seems that all the threads are waiting for
the GC (garbage collector). The top of the stack for most threads read:

NTDLL!NtWaitForSingleObject
KERNEL32!WaitForSingleObject
mscorwks!GCHeap::WaitUntilGCComplete

Is this normal behavior? Is it safe to assume that the Garbage
Collector has already attempted to clean up things, and the processes
are just waiting for it to finish?

Confused,

Jeff Cheung

Ivan Brugiolo [MSFT]

unread,
Jul 2, 2003, 1:56:09 PM7/2/03
to
There should be one thread performing garbage collection,
and all the others executing managed code are expected to be waiting
in selected "safe" points till the collection has completed.

Could you just report the output of '~*kb' ?

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"Cheung, Jeffrey Jing-Yen" <jin...@emailREMOVEMEuser.net> wrote in message

news:#6X8OXLQ...@tk2msftngp13.phx.gbl...

Paul Wehland

unread,
Jul 3, 2003, 2:43:16 PM7/3/03
to
We encountered the same hang. Although I do not have the entire
stack, I do have the threads that were deadlocking:

. 0 Id: 36c.170 Suspend: 1 Teb: 7ffde000 Unfrozen
ChildEBP RetAddr Args to Child
0012f660 77e962ce 000000cc 00000000 00000000 ntdll!NtWriteFile+0xb
0012f6cc 0044291e 000000cc 0012f6f4 00000000 KERNEL32!WriteFile+0x111
0012f700 00442eb1 77ea9d54 00442e06 00000000
aspnet_wp!CSyncPipeManager::Ping+0x4c
0012f708 00442e06 00000000 00fcdcb4 7ffdf000 aspnet_wp!TestPipes+0x27
0012ff60 0044408d 00000000 002f3770 002f41f0 aspnet_wp!wmain+0x3b7
0012ffc0 77ea847c 7ffd6000 00fcdcb4 7ffdf000
aspnet_wp!wmainCRTStartup+0x143
0012fff0 00000000 00443f4a 00000000 000000c8
KERNEL32!BaseProcessStart+0x3d

1 Id: 36c.700 Suspend: 1 Teb: 7ffdd000 Unfrozen
ChildEBP RetAddr Args to Child
00b6f910 77e96156 000000c8 00000000 00000000 ntdll!ZwReadFile+0xb
00b6f984 0044288e 000000c8 00501e90 00000019 KERNEL32!ReadFile+0x181
00b6f9b8 004425ef 00502d38 00501e90 00000000
aspnet_wp!CSyncPipeManager::SendSyncMessage+0x7a
00b6f9e0 004439ea 0000000b 00000001 00000002
aspnet_wp!CSyncPipeManager::GetDataFromIIS+0xab
00b6fa04 00dcabad 00504e98 00000002 0cfd5a5c
aspnet_wp!PMCallISAPI+0x36


Our workaround was to run the appliction at a higher protection level
than Low. The interesting thing is that aspnet_wp.exe did not
deadlock when a debugger was attached, we had to hang first then
attach cdb to discover the pipe that aspnet_wp was deadlocking on. I
do not think Microsoft has a fix for this bug yet, but you can
workaround it by not running at low protection level.

--
Paul Wehland

"Ivan Brugiolo [MSFT]" <ivan...@online.microsoft.com> wrote in message news:<u7KX3MMQ...@tk2msftngp13.phx.gbl>...

Cheung, Jeffrey Jing-Yen

unread,
Jul 4, 2003, 1:54:41 AM7/4/03
to
Paul,

The site is actaully running in High Isolation mode now, and it still
deadlocks unfortunately... :(

Cheung, Jeffrey Jing-Yen

unread,
Jul 4, 2003, 1:59:07 AM7/4/03
to
Ivan,

I will glady provide the output. I hope you can help me decipher this
as this is my first time *truly* reading debugger info.

0:012> ~*kb

0 Id: 12c.4f4 Suspend: 1 Teb: 7ffde000 Unfrozen


ChildEBP RetAddr Args to Child

0012f6d8 77ea9d94 00000000 0012f6f0 7c349434 NTDLL!NtDelayExecution+0xb
0012f6f8 77ea9d5f 000000fa 00000000 00442df0 KERNEL32!SleepEx+0x32
0012f704 00442df0 000000fa 00000000 0556e8cc KERNEL32!Sleep+0xb
0012ff60 0044408d 00000000 002f3730 002f4110 aspnet_wp!wmain+0x3a1
0012ffc0 77ea847c 7ff47000 0556e8cc 7ffdf000 aspnet_wp!wmainCRTStartup+0x143


0012fff0 00000000 00443f4a 00000000 000000c8 KERNEL32!BaseProcessStart+0x3d

1 Id: 12c.470 Suspend: 1 Teb: 7ffdd000 Unfrozen


ChildEBP RetAddr Args to Child

00b6ff54 77ea9b35 00000098 00000000 00b6ff74 NTDLL!NtWaitForSingleObject+0xb
00b6ff7c 77e8b32b 00000098 00001388 00000000
KERNEL32!WaitForSingleObjectEx+0x71
00b6ff8c 792e9c0d 00000098 00001388 0043005c
KERNEL32!WaitForSingleObject+0xf
00b6ffb4 77e8b2d8 00000000 0043005c 006e006f
mscorwks!ThreadpoolMgr::CompletionPortThreadStart+0x127
00b6ffec 00000000 792e9ae6 00000000 00000000 KERNEL32!BaseThreadStart+0x52

2 Id: 12c.934 Suspend: 1 Teb: 7ffdc000 Unfrozen


ChildEBP RetAddr Args to Child

00c6ff2c 77ea9d94 00000000 00c6ff44 00000000 NTDLL!NtDelayExecution+0xb
00c6ff4c 77ea9d5f 000001f4 00000000 792e9ebe KERNEL32!SleepEx+0x32
00c6ff58 792e9ebe 000001f4 fbf5f0f0 00000279 KERNEL32!Sleep+0xb
00c6ffb4 77e8b2d8 00000000 00000018 00003000
mscorwks!ThreadpoolMgr::GateThreadStart+0x54
00c6ffec 00000000 792e9e6a 00000000 00000000 KERNEL32!BaseThreadStart+0x52

3 Id: 12c.4fc Suspend: 1 Teb: 7ffdb000 Unfrozen


ChildEBP RetAddr Args to Child

00d6ff3c 77ea9b35 000000b8 00000000 00d6ff5c NTDLL!NtWaitForSingleObject+0xb
00d6ff64 77e8b32b 000000b8 00000064 00000000
KERNEL32!WaitForSingleObjectEx+0x71
00d6ff74 00443020 000000b8 00000064 7c34940f
KERNEL32!WaitForSingleObject+0xf
00d6ff80 7c34940f 00000000 0012f430 77fc9682 aspnet_wp!DoPingThread+0x10
00d6ffb4 77e8b2d8 002f43e0 0012f430 77fc9682 msvcr71!_endthread+0xaa
00d6ffec 00000000 7c3493a3 002f43e0 00000000 KERNEL32!BaseThreadStart+0x52

4 Id: 12c.670 Suspend: 1 Teb: 7ffda000 Unfrozen


ChildEBP RetAddr Args to Child

04f0fecc 77ea9d00 00000003 04f0fef4 00000001
NTDLL!ZwWaitForMultipleObjects+0xb
04f0ff1c 77ea9c13 04f0fef4 00000001 00000000
KERNEL32!WaitForMultipleObjectsEx+0xea
04f0ff34 791d25d5 00000003 04f0ff58 00000000
KERNEL32!WaitForMultipleObjects+0x17
04f0ff9c 791d254c 00000000 00130178 00130178
mscorwks!DebuggerRCThread::MainLoop+0x90
04f0ffac 791d4d50 04f0ffec 77e8b2d8 00e10008
mscorwks!DebuggerRCThread::ThreadProc+0x68
04f0ffb4 77e8b2d8 00e10008 00130178 00130178
mscorwks!DebuggerRCThread::ThreadProcStatic+0xb
04f0ffec 00000000 791d4d45 00e10008 00000000 KERNEL32!BaseThreadStart+0x52

5 Id: 12c.838 Suspend: 1 Teb: 7ffd9000 Unfrozen


ChildEBP RetAddr Args to Child

070aff28 77ea9b35 00000140 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
070aff50 77e8b32b 00000140 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
070aff60 791bbe7a 00000140 ffffffff 070affb4
KERNEL32!WaitForSingleObject+0xf
070aff70 791bbea4 00000140 0014fcf8 00000000
mscorwks!WaitForFinalizerEvent+0xde
070affb4 77e8b2d8 00000000 0014fcf8 00000000
mscorwks!GCHeap::FinalizerThreadStart+0x96
070affec 00000000 791d0020 00000000 00000000 KERNEL32!BaseThreadStart+0x52

6 Id: 12c.59c Suspend: 1 Teb: 7ffd7000 Unfrozen


ChildEBP RetAddr Args to Child

0753ff80 77ea9d94 00000001 0753ff98 000003e8 NTDLL!NtDelayExecution+0xb
0753ffa0 792ea483 000003e8 00000001 00b6faf4 KERNEL32!SleepEx+0x32
0753ffb4 77e8b2d8 00000000 00b6faf4 00b6fadc
mscorwks!ThreadpoolMgr::TimerThreadStart+0x30
0753ffec 00000000 792ea453 00000000 00000000 KERNEL32!BaseThreadStart+0x52

7 Id: 12c.a54 Suspend: 1 Teb: 7ffd6000 Unfrozen


ChildEBP RetAddr Args to Child

0785ff54 77ea9b35 00000098 00000000 0785ff74 NTDLL!NtWaitForSingleObject+0xb
0785ff7c 77e8b32b 00000098 00001388 00000000
KERNEL32!WaitForSingleObjectEx+0x71
0785ff8c 792e9c0d 00000098 00001388 00000000
KERNEL32!WaitForSingleObject+0xf
0785ffb4 77e8b2d8 00000000 00000000 002f43d8
mscorwks!ThreadpoolMgr::CompletionPortThreadStart+0x127
0785ffec 00000000 792e9ae6 001b2688 00000000 KERNEL32!BaseThreadStart+0x52

8 Id: 12c.948 Suspend: 1 Teb: 7ffd4000 Unfrozen


ChildEBP RetAddr Args to Child

081dff88 751a44ba 00000002 081dffac 00000001
NTDLL!ZwWaitForMultipleObjects+0xb
081dffb4 77e8b2d8 00000000 7ffddbf8 00b6dd54 NETAPI32!NetbiosWaiter+0x71
081dffec 00000000 751a4449 00000000 00000000 KERNEL32!BaseThreadStart+0x52

9 Id: 12c.a5c Suspend: 1 Teb: 7ffaf000 Unfrozen


ChildEBP RetAddr Args to Child

082dff58 77ea9b35 000002f4 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
082dff80 77e8b32b 000002f4 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
*** ERROR: Symbol file could not be found. Defaulted to export symbols
for oledb32.dll -
082dff90 1f8bc562 000002f4 ffffffff 0000009e
KERNEL32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be
wrong.
082dffb4 77e8b2d8 07e335d0 0000009e 0052005c
oledb32!DllGetClassObject+0x60a8
082dffec 00000000 1f8bc357 07e335d0 00000000 KERNEL32!BaseThreadStart+0x52

10 Id: 12c.990 Suspend: 1 Teb: 7ffae000 Unfrozen


ChildEBP RetAddr Args to Child

083dff50 77ea9b35 000002ec 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
083dff78 77e8b32b 000002ec ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
083dff88 1f8bc554 000002ec ffffffff 07e33118
KERNEL32!WaitForSingleObject+0xf
WARNING: Stack unwind information not available. Following frames may be
wrong.
083dffb4 77e8b2d8 07e33118 0000009e 0052005c
oledb32!DllGetClassObject+0x609a
083dffec 00000000 1f8bc2cf 07e33118 00000000 KERNEL32!BaseThreadStart+0x52

11 Id: 12c.66c Suspend: 1 Teb: 7ffad000 Unfrozen


ChildEBP RetAddr Args to Child

084dfd54 77ea9b35 00000300 00000000 084dfd74 NTDLL!NtWaitForSingleObject+0xb
084dfd7c 77e8b32b 00000300 00002710 00000000
KERNEL32!WaitForSingleObjectEx+0x71
084dfd8c 7878d255 00000300 00002710 00b6ecd8
KERNEL32!WaitForSingleObject+0xf
084dffb4 77e8b2d8 001e8594 00b6ecd8 00000200 comsvcs!PingThread+0xf5
084e00b4 787e8038 8b0d7400 ce8b2056 92b415ff KERNEL32!BaseThreadStart+0x52

# 12 Id: 12c.858 Suspend: 1 Teb: 7ffac000 Unfrozen


ChildEBP RetAddr Args to Child

0876ff2c 00441b9b 793f3b70 00000000 00441240 NTDLL!DbgBreakPoint
0876ff44 79e8cb3d 00970d30 00000000 0000000c
aspnet_wp!CAsyncPipeManager::ProcessCompletion+0xfc
0876ff88 792e9c69 00000000 0000000c 00a281d8
aspnet_isapi!CorThreadPoolCompletionCallback+0x3c
0876ffb4 77e8b2d8 00000001 00000000 002f43d8
mscorwks!ThreadpoolMgr::CompletionPortThreadStart+0x183
0876ffec 00000000 792e9ae6 0019cb40 00000000 KERNEL32!BaseThreadStart+0x52

13 Id: 12c.630 Suspend: 1 Teb: 7ffaa000 Unfrozen


ChildEBP RetAddr Args to Child

0b0cff6c 77ea9d94 00000001 0b0cff84 00000000 NTDLL!NtDelayExecution+0xb
0b0cff8c 792ea315 ffffffff 00000001 00b6fb44 KERNEL32!SleepEx+0x32
0b0cffb4 77e8b2d8 00000000 00b6fb44 00b6fb40
mscorwks!ThreadpoolMgr::WaitThreadStart+0x22
0b0cffec 00000000 792ea2f3 0015e010 00000000 KERNEL32!BaseThreadStart+0x52

14 Id: 12c.8b0 Suspend: 1 Teb: 7ffab000 Unfrozen


ChildEBP RetAddr Args to Child

0b44ff30 77abaf78 0b44ff54 00000000 00000000 USER32!NtUserGetMessage+0xb
0b44ff70 77abaec6 77ea9ac4 0021c868 00000102
OLE32!CDllHost::STAWorkerLoop+0x40
0b44ff8c 77abae01 77ab47ac 77b2c318 77a50000
OLE32!CDllHost::WorkerThread+0xc2
0b44ff90 77ab47ac 77b2c318 77a50000 0021c868 OLE32!DLLHostThreadEntry+0x9
0b44ffa8 77ab4704 0043005f 0061006c 77e8b2d8
OLE32!CRpcThread::WorkerLoop+0x22
0b44ffb4 77e8b2d8 0021c868 0043005f 0061006c
OLE32!CRpcThreadCache::RpcWorkerThreadEntry+0x1a
0b44ffec 00000000 77ab46ea 0021c868 00000000 KERNEL32!BaseThreadStart+0x52

15 Id: 12c.8dc Suspend: 1 Teb: 7ffa4000 Unfrozen


ChildEBP RetAddr Args to Child

0ed8f5b4 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
0ed8f5dc 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
0ed8f5ec 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
0ed8f5f8 79201b76 07e878ac 084f0928 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
0ed8f608 791bfbf1 792035e0 00000404 0ed8f6cc
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
0ed8f6bc 00ddb999 00213450 0ed8f6f0 08d16a78
mscorwks!ComPlusToComWorker+0x1cc
WARNING: Frame IP not in any known module. Following frames may be wrong.
0ed8f764 084ec5c7 16f8b58c 07e0d849 00000006 0xddb999
0ed8f7b0 07e0d50f 04f9e394 16f76e20 16f89900 0x84ec5c7
0ed8f7dc 07e0d4b6 084ec33a 16f76e20 16f89900 0x7e0d50f
0ed8f820 084ec230 16f898e8 08cb9ee5 16f76e20 0x7e0d4b6
0ed8f860 0abd0f1b 050265ac 000986ca 12879b2c 0x84ec230
0ed8f8b8 0abd0a2d 000986ca 00000000 05835ce8 0xabd0f1b
0ed8f940 0abd0866 00000000 00000000 00213450 0xabd0a2d
0ed8f964 0c49bb5a 00000000 00000000 0ed8f998 0xabd0866
0ed8f980 7931bce6 0ed8f9d0 00000001 0ed8f9a4 0xc49bb5a
0ed8fb50 07431d72 0ed8fb5c 04f992f4 00000001
mscorwks!COMMember::InvokeMethod+0x95a
00000002 00000000 00000000 00000000 00000000 0x7431d72

16 Id: 12c.938 Suspend: 1 Teb: 7ffd5000 Unfrozen


ChildEBP RetAddr Args to Child

0ce6fe24 77d58edc 000001d8 0ce6ff54 00000000
NTDLL!NtReplyWaitReceivePortEx+0xb
0ce6ff74 77d587a2 77d422ae 0015fa38 77fc8d32
rpcrt4!LRPC_ADDRESS::ReceiveLotsaCalls+0x74
0ce6ff78 77d422ae 0015fa38 77fc8d32 002f0a18
rpcrt4!RecvLotsaCallsWrapper+0x9
0ce6ffa8 77d3f157 0c7e0a30 0ce6ffec 77e8b2d8
rpcrt4!BaseCachedThreadRoutine+0x4f
0ce6ffb4 77e8b2d8 0c7f8ef0 77fc8d32 002f0a18 rpcrt4!ThreadStartRoutine+0x18
0ce6ffec 00000000 77d3f13f 0c7f8ef0 00000000 KERNEL32!BaseThreadStart+0x52

17 Id: 12c.510 Suspend: 1 Teb: 7ffa8000 Unfrozen


ChildEBP RetAddr Args to Child

0e18ed44 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
0e18ed6c 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
0e18ed7c 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
0e18ed88 79201b76 00000001 08ddf3a8 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
0e18ed98 792ccbef 0c8c2ee8 79bb57b4 00100000
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
0e18edb0 792ce506 79bb57b4 00100000 0e18edd0
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x5ae
0e18edc0 791b3c64 00000014 00000002 0e18edf0 mscorwks!GCHeap::Alloc+0x12
0e18edd0 791d51e1 00000014 00000000 00040000 mscorwks!Alloc+0x3a
0e18edf0 7920a2af 79bb57b4 0e18ee60 0e18ee3c
mscorwks!FastAllocateObject+0x25
0e18ee00 7920a309 79bb57b4 08ddf3a8 0e18ee60
mscorwks!ComObject::CreateComObjectRef+0x23
0e18ee3c 7920a257 00000000 00000000 00000000
mscorwks!COMInterfaceMarshaler::CreateObjectRef+0x4a
0e18ee4c 7920910f 0e18f610 0e18f610 00000000
mscorwks!COMInterfaceMarshaler::FindOrCreateObjectRef+0x43
0e18f2ec 7923522a 09488c78 00000000 00000000
mscorwks!GetObjectRefFromComIP+0x1a3
0e18f524 7920c87d 0e18f610 7920c846 00000004
mscorwks!InterfaceMarshaler::ConvertSpaceNativeToCom+0x3b
0e18f52c 7920c846 00000004 084f0928 00000000
mscorwks!InterfaceMarshaler::UnmarshalComToNativeByrefOut+0x8
0e18f5f4 791bfc43 084f094c 0e18f690 0e18f698 mscorwks!RunML+0x8d9
0e18f6bc 00ddb999 08ddf3a8 0e18f6f0 0c80e0a0
mscorwks!ComPlusToComWorker+0x2c2
WARNING: Frame IP not in any known module. Following frames may be wrong.
0e18f764 084ec5c7 17f0dfdc 07e0d849 00000006 0xddb999
0e18f7b0 07e0d50f 04f9e394 16e5bf0c 16e6b5b0 0x84ec5c7
0e18f7dc 07e0d4b6 084ec33a 16e5bf0c 17f0df30 0x7e0d50f

18 Id: 12c.3ec Suspend: 1 Teb: 7ffd8000 Unfrozen


ChildEBP RetAddr Args to Child

0d8cf450 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
0d8cf478 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
0d8cf488 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
0d8cf494 79201b76 07e76d74 0db97868 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
0d8cf4a4 7920279d 00000003 0c89c644 00136ae0
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
0d8cf4d0 7920c519 00000001 00000000 0c89c610 mscorwks!SafeRelease+0x7b
0d8cf4e0 7920c5d5 001ec22c 0c89c610 001ec210
mscorwks!ComPlusWrapper::ReleaseAllInterfaces+0x22
0d8cf718 7920c466 0c89c610 0c89c610 79211ffb
mscorwks!ComPlusWrapper::ReleaseAllInterfacesCallBack+0x12b
0d8cf724 79211ffb 0d8cfa24 16eb723c 0db97868
mscorwks!ComPlusWrapper::Cleanup+0x16
0d8cf740 7921203d 16eb723c 0d8cf778 16e783b4
mscorwks!ComPlusWrapper::ExternalRelease+0x74
0d8cf750 07433e52 0d8cf75c 16eb723c 79bb57b4
mscorwks!Interop::ReleaseComObject+0x49
WARNING: Frame IP not in any known module. Following frames may be wrong.
16e783b4 00000000 00000000 00000000 12ccd224 0x7433e52

19 Id: 12c.798 Suspend: 1 Teb: 7ffa6000 Unfrozen


ChildEBP RetAddr Args to Child

0eefecf0 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
0eefed18 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
0eefed28 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
0eefed34 79201b76 00000001 0dac5468 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
0eefed44 792ccbef 0c8ead90 79bb57b4 00100000
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
0eefed5c 792ce506 79bb57b4 00100000 0eefed7c
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x5ae
0eefed6c 791b3c64 00000014 00000002 0eefed9c mscorwks!GCHeap::Alloc+0x12
0eefed7c 791d51e1 00000014 00000000 00040000 mscorwks!Alloc+0x3a
0eefed9c 7920a2af 79bb57b4 0eefee0c 0eefede8
mscorwks!FastAllocateObject+0x25
0eefedac 7920a309 79bb57b4 0dac5468 0eefee0c
mscorwks!ComObject::CreateComObjectRef+0x23
0eefede8 7920a257 00000000 00000000 00000000
mscorwks!COMInterfaceMarshaler::CreateObjectRef+0x4a
0eefedf8 7920910f 0eeff5bc 0eeff5bc 00000000
mscorwks!COMInterfaceMarshaler::FindOrCreateObjectRef+0x43
0eeff298 7923522a 09467738 00000000 00000000
mscorwks!GetObjectRefFromComIP+0x1a3
0eeff4d0 7920c87d 0eeff5bc 7920c846 00000004
mscorwks!InterfaceMarshaler::ConvertSpaceNativeToCom+0x3b
0eeff4d8 7920c846 00000004 084f0928 00000000
mscorwks!InterfaceMarshaler::UnmarshalComToNativeByrefOut+0x8
0eeff5a0 791bfc43 084f094c 0eeff63c 0eeff644 mscorwks!RunML+0x8d9
0eeff668 00ddb999 0dac5468 0eeff69c 0c8207d0
mscorwks!ComPlusToComWorker+0x2c2
WARNING: Frame IP not in any known module. Following frames may be wrong.
0eeff710 084ec5c7 16e35bac 07e0d849 00000006 0xddb999
0eeff75c 07e0d50f 04f9e394 050269a0 16f2bbd8 0x84ec5c7
0eeff788 07e0d4b6 084ec33a 050269a0 16e35ae0 0x7e0d50f

20 Id: 12c.558 Suspend: 1 Teb: 7ffa7000 Unfrozen


ChildEBP RetAddr Args to Child

0e48ed44 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
0e48ed6c 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
0e48ed7c 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
0e48ed88 79201b76 00000001 08d97dd8 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
0e48ed98 792ccbef 0c8f0cb0 79bb57b4 00100000
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
0e48edb0 792ce506 79bb57b4 00100000 0e48edd0
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x5ae
0e48edc0 791b3c64 00000014 00000002 0e48edf0 mscorwks!GCHeap::Alloc+0x12
0e48edd0 791d51e1 00000014 00000000 00040000 mscorwks!Alloc+0x3a
0e48edf0 7920a2af 79bb57b4 0e48ee60 0e48ee3c
mscorwks!FastAllocateObject+0x25
0e48ee00 7920a309 79bb57b4 08d97dd8 0e48ee60
mscorwks!ComObject::CreateComObjectRef+0x23
0e48ee3c 7920a257 00000000 00000000 00000000
mscorwks!COMInterfaceMarshaler::CreateObjectRef+0x4a
0e48ee4c 7920910f 0e48f610 0e48f610 00000000
mscorwks!COMInterfaceMarshaler::FindOrCreateObjectRef+0x43
0e48f2ec 7923522a 09456ca0 00000000 00000000
mscorwks!GetObjectRefFromComIP+0x1a3
0e48f524 7920c87d 0e48f610 7920c846 00000004
mscorwks!InterfaceMarshaler::ConvertSpaceNativeToCom+0x3b
0e48f52c 7920c846 00000004 084f0928 00000000
mscorwks!InterfaceMarshaler::UnmarshalComToNativeByrefOut+0x8
0e48f5f4 791bfc43 084f094c 0e48f690 0e48f698 mscorwks!RunML+0x8d9
0e48f6bc 00ddb999 08d97dd8 0e48f6f0 0c7c3ca0
mscorwks!ComPlusToComWorker+0x2c2
WARNING: Frame IP not in any known module. Following frames may be wrong.
0e48f764 084ec5c7 17f0eea0 07e0d849 00000006 0xddb999
0e48f7b0 07e0d50f 04f9e394 16e692f4 16e7a778 0x84ec5c7
0e48f7dc 07e0d4b6 084ec33a 16e692f4 17f0edf4 0x7e0d50f

21 Id: 12c.924 Suspend: 1 Teb: 7ffa9000 Unfrozen


ChildEBP RetAddr Args to Child

1573ed44 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
1573ed6c 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
1573ed7c 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
1573ed88 79201b76 00000001 08d91a90 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
1573ed98 792ccbef 0c8c6778 79bb57b4 00100000
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
1573edb0 792ce506 79bb57b4 00100000 1573edd0
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x5ae
1573edc0 791b3c64 00000014 00000002 1573edf0 mscorwks!GCHeap::Alloc+0x12
1573edd0 791d51e1 00000014 00000000 00040000 mscorwks!Alloc+0x3a
1573edf0 7920a2af 79bb57b4 1573ee60 1573ee3c
mscorwks!FastAllocateObject+0x25
1573ee00 7920a309 79bb57b4 08d91a90 1573ee60
mscorwks!ComObject::CreateComObjectRef+0x23
1573ee3c 7920a257 00000000 00000000 00000000
mscorwks!COMInterfaceMarshaler::CreateObjectRef+0x4a
1573ee4c 7920910f 1573f610 1573f610 00000000
mscorwks!COMInterfaceMarshaler::FindOrCreateObjectRef+0x43
1573f2ec 7923522a 07ef23a8 00000000 00000000
mscorwks!GetObjectRefFromComIP+0x1a3
1573f524 7920c87d 1573f610 7920c846 00000004
mscorwks!InterfaceMarshaler::ConvertSpaceNativeToCom+0x3b
1573f52c 7920c846 00000004 084f0928 00000000
mscorwks!InterfaceMarshaler::UnmarshalComToNativeByrefOut+0x8
1573f5f4 791bfc43 084f094c 1573f690 1573f698 mscorwks!RunML+0x8d9
1573f6bc 00ddb999 08d91a90 1573f6f0 0c896a18
mscorwks!ComPlusToComWorker+0x2c2
WARNING: Frame IP not in any known module. Following frames may be wrong.
1573f764 084ec5c7 16dccac4 07e0d849 00000006 0xddb999
1573f7b0 07e0d50f 04f9e394 16e4489c 16ec52f0 0x84ec5c7
1573f7dc 07e0d4b6 084ec33a 16e4489c 16dcca18 0x7e0d50f

22 Id: 12c.894 Suspend: 1 Teb: 7ffa5000 Unfrozen


ChildEBP RetAddr Args to Child

1080ed44 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
1080ed6c 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
1080ed7c 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
1080ed88 79201b76 00000001 0dacd078 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
1080ed98 792ccbef 0c8c88c8 79bb57b4 00100000
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
1080edb0 792ce506 79bb57b4 00100000 1080edd0
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x5ae
1080edc0 791b3c64 00000014 00000002 1080edf0 mscorwks!GCHeap::Alloc+0x12
1080edd0 791d51e1 00000014 00000000 00040000 mscorwks!Alloc+0x3a
1080edf0 7920a2af 79bb57b4 1080ee60 1080ee3c
mscorwks!FastAllocateObject+0x25
1080ee00 7920a309 79bb57b4 0dacd078 1080ee60
mscorwks!ComObject::CreateComObjectRef+0x23
1080ee3c 7920a257 00000000 00000000 00000000
mscorwks!COMInterfaceMarshaler::CreateObjectRef+0x4a
1080ee4c 7920910f 1080f610 1080f610 00000000
mscorwks!COMInterfaceMarshaler::FindOrCreateObjectRef+0x43
1080f2ec 7923522a 07efecc8 00000000 00000000
mscorwks!GetObjectRefFromComIP+0x1a3
1080f524 7920c87d 1080f610 7920c846 00000004
mscorwks!InterfaceMarshaler::ConvertSpaceNativeToCom+0x3b
1080f52c 7920c846 00000004 084f0928 00000000
mscorwks!InterfaceMarshaler::UnmarshalComToNativeByrefOut+0x8
1080f5f4 791bfc43 084f094c 1080f690 1080f698 mscorwks!RunML+0x8d9
1080f6bc 00ddb999 0dacd078 1080f6f0 0c884af8
mscorwks!ComPlusToComWorker+0x2c2
WARNING: Frame IP not in any known module. Following frames may be wrong.
1080f764 084ec5c7 17f24c14 07e0d849 00000006 0xddb999
1080f7b0 07e0d50f 04f9e394 16ea127c 16ead898 0x84ec5c7
1080f7dc 07e0d4b6 084ec33a 16ea127c 17f24b68 0x7e0d50f

23 Id: 12c.584 Suspend: 1 Teb: 7ffa3000 Unfrozen


ChildEBP RetAddr Args to Child

10d3f5b4 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
10d3f5dc 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
10d3f5ec 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
10d3f5f8 79201b76 0944f754 084f0928 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
10d3f608 791bfbf1 792035e0 00000404 10d3f6cc
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
10d3f6bc 00ddb999 08dbfa68 10d3f6f0 0c86b960
mscorwks!ComPlusToComWorker+0x1cc
WARNING: Frame IP not in any known module. Following frames may be wrong.
10d3f764 084ec5c7 16fad150 07e0d849 00000006 0xddb999
10d3f7b0 07e0d50f 04f9e394 16f98810 16fab4c4 0x84ec5c7
10d3f7dc 07e0d4b6 084ec33a 16f98810 16fab4c4 0x7e0d50f
10d3f820 084ec230 16fab4ac 08cb9ee5 16f98810 0x7e0d4b6
10d3f860 0abd0f1b 050265ac 00098371 12e59718 0x84ec230
10d3f8b8 0abd0a2d 00098371 00000000 05a06288 0xabd0f1b
10d3f940 0abd0866 00000000 00000000 08dbfa68 0xabd0a2d
10d3f964 0c49bb5a 00000000 00000000 10d3f998 0xabd0866
10d3f980 7931bce6 10d3f9d0 00000001 10d3f9a4 0xc49bb5a
10d3fb50 07431d72 10d3fb5c 04f992f4 00000001
mscorwks!COMMember::InvokeMethod+0x95a
00000002 00000000 00000000 00000000 00000000 0x7431d72

24 Id: 12c.828 Suspend: 1 Teb: 7ffa2000 Unfrozen


ChildEBP RetAddr Args to Child

10eaed44 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
10eaed6c 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
10eaed7c 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
10eaed88 79201b76 00000001 0da31a90 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
10eaed98 792ccbef 0c8d4108 79bb57b4 00100000
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
10eaedb0 792ce506 79bb57b4 00100000 10eaedd0
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x5ae
10eaedc0 791b3c64 00000014 00000002 10eaedf0 mscorwks!GCHeap::Alloc+0x12
10eaedd0 791d51e1 00000014 00000000 00040000 mscorwks!Alloc+0x3a
10eaedf0 7920a2af 79bb57b4 10eaee60 10eaee3c
mscorwks!FastAllocateObject+0x25
10eaee00 7920a309 79bb57b4 0da31a90 10eaee60
mscorwks!ComObject::CreateComObjectRef+0x23
10eaee3c 7920a257 00000000 00000000 00000000
mscorwks!COMInterfaceMarshaler::CreateObjectRef+0x4a
10eaee4c 7920910f 10eaf610 10eaf610 00000000
mscorwks!COMInterfaceMarshaler::FindOrCreateObjectRef+0x43
10eaf2ec 7923522a 07e85cb0 00000000 00000000
mscorwks!GetObjectRefFromComIP+0x1a3
10eaf524 7920c87d 10eaf610 7920c846 00000004
mscorwks!InterfaceMarshaler::ConvertSpaceNativeToCom+0x3b
10eaf52c 7920c846 00000004 084f0928 00000000
mscorwks!InterfaceMarshaler::UnmarshalComToNativeByrefOut+0x8
10eaf5f4 791bfc43 084f094c 10eaf690 10eaf698 mscorwks!RunML+0x8d9
10eaf6bc 00ddb999 0da31a90 10eaf6f0 0c86b080
mscorwks!ComPlusToComWorker+0x2c2
WARNING: Frame IP not in any known module. Following frames may be wrong.
10eaf764 084ec5c7 16e2ebe8 07e0d849 00000006 0xddb999
10eaf7b0 07e0d50f 04f9e394 16e8d90c 16f1dfe0 0x84ec5c7
10eaf7dc 07e0d4b6 084ec33a 16e8d90c 16e2eb3c 0x7e0d50f

25 Id: 12c.4dc Suspend: 1 Teb: 7ff5f000 Unfrozen


ChildEBP RetAddr Args to Child

11daf260 77ea9d00 00000001 11daf288 00000001
NTDLL!ZwWaitForMultipleObjects+0xb
11daf2b0 77ea9c13 11daf288 00000001 00000000
KERNEL32!WaitForMultipleObjectsEx+0xea
11daf2c8 7924699d 00000001 11daf3ec 00000000
KERNEL32!WaitForMultipleObjects+0x17
11daf500 79237538 00000001 00000000 00000000
mscorwks!Thread::SysSuspendForGC+0x248
11daf518 79238d3d 00000001 00000000 00000000 mscorwks!GCHeap::SuspendEE+0xcf
11daf534 7923a404 00000000 00000000 086522b4
mscorwks!GCHeap::GarbageCollectGeneration+0x103
11daf564 792ce558 793dc350 00000040 00000000
mscorwks!gc_heap::allocate_more_space+0x13a
11daf580 791b3c64 00000040 00000002 11daf5b0 mscorwks!GCHeap::Alloc+0x64
11daf590 791d51e1 00000040 00000000 00040000 mscorwks!Alloc+0x3a
11daf5b0 791d522f 086522b4 16fe1a50 16fe3428
mscorwks!FastAllocateObject+0x25
11daf61c 084e51d1 16fe335c 16fe336c 16fe1ea0 mscorwks!JIT_NewFast+0x2c
WARNING: Frame IP not in any known module. Following frames may be wrong.
00000003 00000000 00000000 00000000 00000000 0x84e51d1

26 Id: 12c.9c4 Suspend: 1 Teb: 7ff5e000 Unfrozen


ChildEBP RetAddr Args to Child

11f4f2b4 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
11f4f2dc 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
11f4f2ec 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
11f4f2f8 79201b76 16fde49c 08dbe530 16fde4f4
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
11f4f308 792974a0 16fde4cc 791b42f0 11f4f584
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
11f4f328 792978ad 16fde49c 16fde4cc 16fde4f4
mscorwks!OnHijackScalarWorker+0x35
04f9d990 00000000 00000000 00000000 01010100
mscorwks!OnHijackScalarTripThread+0xd

27 Id: 12c.774 Suspend: 1 Teb: 7ff5d000 Unfrozen


ChildEBP RetAddr Args to Child

122bf498 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
122bf4c0 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
122bf4d0 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
122bf4dc 79201b76 00000001 08de3550 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
122bf4ec 792ccbef 00e031e8 00000074 00000000
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
122bf504 792ce506 00000074 00000000 122bf524
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x5ae
122bf514 791b3c64 00000074 00000000 122bf544 mscorwks!GCHeap::Alloc+0x12
122bf524 791dbe2f 00000074 00000000 00000000 mscorwks!Alloc+0x3a
122bf544 791dc002 00e031e8 0000001a 00000000
mscorwks!FastAllocatePrimitiveArray+0x45
122bf5c4 084e1c81 122bf670 16f0b278 0000001a mscorwks!JIT_NewArr1+0xbb
WARNING: Frame IP not in any known module. Following frames may be wrong.
122bf670 084eca17 00000000 0ca66170 0ca66170 0x84e1c81
122bf71c 084ec757 00000003 16e6218c 00000006 0x84eca17
122bf764 084ec5c7 16e6218c 07e0d849 00000006 0x84ec757
122bf7b0 07e0d50f 04f9e394 16e8ed9c 16ee5a80 0x84ec5c7
122bf7dc 07e0d4b6 084ec33a 16e8ed9c 16e620e0 0x7e0d50f
122bf820 08664617 16ee5a68 08cb9f99 16e8ed9c 0x7e0d4b6
122bf860 0abd0f1b 050265ac 000981e1 14641038 0x8664617
122bf8b8 0abd0a2d 000981e1 00000000 05a0fae4 0xabd0f1b
122bf940 0abd0866 00000000 00000000 08de3550 0xabd0a2d
122bf964 0c49bb5a 00000000 00000000 122bf998 0xabd0866

28 Id: 12c.a04 Suspend: 1 Teb: 7ff5c000 Unfrozen


ChildEBP RetAddr Args to Child

123bed4c 77eabc68 792b9f06 123bed74 792ccb90 NTDLL!ZwYieldExecution+0xb
123bed50 792b9f06 123bed74 792ccb90 00000000 KERNEL32!SwitchToThread+0x6
123bed58 792ccb90 00000000 0ca28ba0 79bb57b4 mscorwks!__SwitchToThread+0x19
123bed74 792ce506 79bb57b4 00100000 123bed94
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x54f
123bed84 791b3c64 00000014 00000002 123bedb4 mscorwks!GCHeap::Alloc+0x12
123bed94 791d51e1 00000014 00000000 00040000 mscorwks!Alloc+0x3a
123bedb4 7920a2af 79bb57b4 123bee24 123bee00
mscorwks!FastAllocateObject+0x25
123bedc4 7920a309 79bb57b4 08dc6b28 123bee24
mscorwks!ComObject::CreateComObjectRef+0x23
123bee00 7920a257 00000000 00000000 00000000
mscorwks!COMInterfaceMarshaler::CreateObjectRef+0x4a
123bee10 7920910f 123bf5d4 123bf5d4 00000000
mscorwks!COMInterfaceMarshaler::FindOrCreateObjectRef+0x43
123bf2b0 7923522a 07e3e2e8 79b7c364 00000000
mscorwks!GetObjectRefFromComIP+0x1a3
123bf4e8 7920c87d 123bf5d4 7920c846 00000004
mscorwks!InterfaceMarshaler::ConvertSpaceNativeToCom+0x3b
123bf4f0 7920c846 00000004 00ddbf78 00000000
mscorwks!InterfaceMarshaler::UnmarshalComToNativeByrefOut+0x8
123bf5b8 791bfc43 00ddbf9a 123bf654 123bf65c mscorwks!RunML+0x8d9
123bf680 00ddb999 08dc6b28 123bf6b4 0c9d0060
mscorwks!ComPlusToComWorker+0x2c2
WARNING: Frame IP not in any known module. Following frames may be wrong.
16fd68d4 00000000 00000000 0c8f0870 40020000 0xddb999

29 Id: 12c.7a8 Suspend: 1 Teb: 7ff5b000 Unfrozen


ChildEBP RetAddr Args to Child

13acf578 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
13acf5a0 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
13acf5b0 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
13acf5bc 79201b76 0945afcc 00ddbf78 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
13acf5cc 791bfbf1 792035e0 00000404 13acf690
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
13acf680 00ddb999 08d57fe8 13acf6b4 0c996560
mscorwks!ComPlusToComWorker+0x1cc
WARNING: Frame IP not in any known module. Following frames may be wrong.
16ee4270 00000000 00000000 0c923948 40020000 0xddb999

30 Id: 12c.628 Suspend: 1 Teb: 7ff5a000 Unfrozen


ChildEBP RetAddr Args to Child

18c9f5b4 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
18c9f5dc 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
18c9f5ec 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
18c9f5f8 79201b76 0945989c 084f0928 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
18c9f608 791bfbf1 792035e0 00000404 18c9f6cc
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
18c9f6bc 00ddb999 0c9437b8 18c9f6f0 0c9b18a8
mscorwks!ComPlusToComWorker+0x1cc
WARNING: Frame IP not in any known module. Following frames may be wrong.
18c9f764 084ec5c7 16fdc298 07e0d849 00000006 0xddb999
18c9f7b0 07e0d50f 04f9e394 16f05dc4 16fda60c 0x84ec5c7
18c9f7dc 07e0d4b6 084ec33a 16f05dc4 16fda60c 0x7e0d50f
18c9f820 08664617 16fda5f4 08cb9f99 16f05dc4 0x7e0d4b6
18c9f860 0abd0f1b 050265ac 000980ee 14a460a0 0x8664617
18c9f8b8 0abd0a2d 000980ee 00000000 05a1eda0 0xabd0f1b
18c9f940 0abd0866 00000000 00000000 0c9437b8 0xabd0a2d
18c9f964 0c49bb5a 00000000 00000000 18c9f998 0xabd0866
18c9f980 7931bce6 18c9f9d0 00000001 18c9f9a4 0xc49bb5a
18c9fb50 07431d72 18c9fb5c 04f992f4 00000001
mscorwks!COMMember::InvokeMethod+0x95a
00000002 00000000 00000000 00000000 00000000 0x7431d72

31 Id: 12c.548 Suspend: 1 Teb: 7ff59000 Unfrozen


ChildEBP RetAddr Args to Child

18d9ed44 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
18d9ed6c 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
18d9ed7c 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
18d9ed88 79201b76 00000001 0db5e008 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
18d9ed98 792ccbef 0c8f67a0 79bb57b4 00100000
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
18d9edb0 792ce506 79bb57b4 00100000 18d9edd0
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x5ae
18d9edc0 791b3c64 00000014 00000002 18d9edf0 mscorwks!GCHeap::Alloc+0x12
18d9edd0 791d51e1 00000014 00000000 00040000 mscorwks!Alloc+0x3a
18d9edf0 7920a2af 79bb57b4 18d9ee60 18d9ee3c
mscorwks!FastAllocateObject+0x25
18d9ee00 7920a309 79bb57b4 0db5e008 18d9ee60
mscorwks!ComObject::CreateComObjectRef+0x23
18d9ee3c 7920a257 00000000 00000000 00000000
mscorwks!COMInterfaceMarshaler::CreateObjectRef+0x4a
18d9ee4c 7920910f 18d9f610 18d9f610 00000000
mscorwks!COMInterfaceMarshaler::FindOrCreateObjectRef+0x43
18d9f2ec 7923522a 07e3c778 00000000 00000000
mscorwks!GetObjectRefFromComIP+0x1a3
18d9f524 7920c87d 18d9f610 7920c846 00000004
mscorwks!InterfaceMarshaler::ConvertSpaceNativeToCom+0x3b
18d9f52c 7920c846 00000004 084f0928 00000000
mscorwks!InterfaceMarshaler::UnmarshalComToNativeByrefOut+0x8
18d9f5f4 791bfc43 084f094c 18d9f690 18d9f698 mscorwks!RunML+0x8d9
18d9f6bc 00ddb999 0db5e008 18d9f6f0 0db62e70
mscorwks!ComPlusToComWorker+0x2c2
WARNING: Frame IP not in any known module. Following frames may be wrong.
18d9f764 084ec5c7 16dcc618 07e0d849 00000006 0xddb999
18d9f7b0 07e0d50f 04f9e394 16ea4b6c 16ec2f44 0x84ec5c7
18d9f7dc 07e0d4b6 084ec33a 16ea4b6c 16dcc56c 0x7e0d50f

32 Id: 12c.a20 Suspend: 1 Teb: 7ffa1000 Unfrozen


ChildEBP RetAddr Args to Child

18e9f450 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
18e9f478 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
18e9f488 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
18e9f494 79201b76 07e41064 0ca33f68 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
18e9f4a4 7920279d 00000003 0c8b158c 00136ae0
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
18e9f4d0 7920c519 00000001 00000000 0c8b1558 mscorwks!SafeRelease+0x7b
18e9f4e0 7920c5d5 001ec22c 0c8b1558 001ec210
mscorwks!ComPlusWrapper::ReleaseAllInterfaces+0x22
18e9f718 7920c466 0c8b1558 0c8b1558 79211ffb
mscorwks!ComPlusWrapper::ReleaseAllInterfacesCallBack+0x12b
18e9f724 79211ffb 18e9fa24 16ee65c4 0ca33f68
mscorwks!ComPlusWrapper::Cleanup+0x16
18e9f740 7921203d 16ee65c4 18e9f778 16e6a550
mscorwks!ComPlusWrapper::ExternalRelease+0x74
18e9f750 07433e52 18e9f75c 16ee65c4 79bb57b4
mscorwks!Interop::ReleaseComObject+0x49
WARNING: Frame IP not in any known module. Following frames may be wrong.
16e6a550 00000000 00000000 00000000 14e44acc 0x7433e52

33 Id: 12c.454 Suspend: 1 Teb: 7ff58000 Unfrozen


ChildEBP RetAddr Args to Child

168cf2c4 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
168cf2ec 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
168cf2fc 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
168cf308 79201b76 00000001 0ca448d8 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
168cf318 792ccbef 00000004 00000018 00000000
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
168cf330 792ce506 00000018 00000000 168cf350
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x5ae
168cf340 791b3c64 00000018 00000000 168cf36c mscorwks!GCHeap::Alloc+0x12
168cf350 791f0ce4 00000018 00000000 00000000 mscorwks!Alloc+0x3a
168cf36c 791b29e4 00000004 00000003 791dd109
mscorwks!SlowAllocateString+0x26
168cf378 791dd109 00000003 168cf3b4 168cf5c4
mscorwks!JIT_TrialAlloc::GenAllocArray+0x29c
168cf388 791f3862 168cf476 00000003 168c0047
mscorwks!COMString::NewString+0x25
168cf5c4 791f38ed 000000c4 00000001 04f1b704 mscorwks!Int32ToDecStr+0xc2
168cf65c 00e0b0bc ffffffff 04f1b68c 00000000
mscorwks!COMNumber::FormatInt32+0xaa
WARNING: Frame IP not in any known module. Following frames may be wrong.
168cf6d0 07e0ecc3 168cf6f0 c8b522bb 11ce5cf3 0xe0b0bc
168cf754 07e0d50f 04f9e394 050265ac 14f9a480 0x7e0ecc3
168cf780 07e0d4b6 07e0bec3 050265ac 00000000 0x7e0d50f
168cf7c0 07e0bdce 00000000 17f3e34c 050265ac 0x7e0d4b6
16f6be80 16f6c010 16f6c050 00000000 086522b4 0x7e0bdce
16f6bfd0 04f9fdc0 04f9fdc0 00000000 00000000 0x16f6c010
086522b4 00000040 08641210 00000000 00120000 0x4f9fdc0

34 Id: 12c.230 Suspend: 1 Teb: 7ff57000 Unfrozen


ChildEBP RetAddr Args to Child

169ced08 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
169ced30 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
169ced40 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
169ced4c 79201b76 00000001 0ca07688 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
169ced5c 792ccbef 0c8c8e18 79bb57b4 00100000
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
169ced74 792ce506 79bb57b4 00100000 169ced94
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x5ae
169ced84 791b3c64 00000014 00000002 169cedb4 mscorwks!GCHeap::Alloc+0x12
169ced94 791d51e1 00000014 00000000 00040000 mscorwks!Alloc+0x3a
169cedb4 7920a2af 79bb57b4 169cee24 169cee00
mscorwks!FastAllocateObject+0x25
169cedc4 7920a309 79bb57b4 0ca07688 169cee24
mscorwks!ComObject::CreateComObjectRef+0x23
169cee00 7920a257 00000000 00000000 00000000
mscorwks!COMInterfaceMarshaler::CreateObjectRef+0x4a
169cee10 7920910f 169cf5d4 169cf5d4 00000000
mscorwks!COMInterfaceMarshaler::FindOrCreateObjectRef+0x43
169cf2b0 7923522a 07e33ec8 79b7c364 00000000
mscorwks!GetObjectRefFromComIP+0x1a3
169cf4e8 7920c87d 169cf5d4 7920c846 00000004
mscorwks!InterfaceMarshaler::ConvertSpaceNativeToCom+0x3b
169cf4f0 7920c846 00000004 00ddbf78 00000000
mscorwks!InterfaceMarshaler::UnmarshalComToNativeByrefOut+0x8
169cf5b8 791bfc43 00ddbf9a 169cf654 169cf65c mscorwks!RunML+0x8d9
169cf680 00ddb999 0ca07688 169cf6b4 0ca3a060
mscorwks!ComPlusToComWorker+0x2c2
WARNING: Frame IP not in any known module. Following frames may be wrong.
16f7a9cc 00000000 00000000 00000000 00000000 0xddb999

35 Id: 12c.810 Suspend: 1 Teb: 7ff56000 Unfrozen


ChildEBP RetAddr Args to Child

17aced44 77ea9b35 00000138 00000000 00000000 NTDLL!NtWaitForSingleObject+0xb
17aced6c 77e8b32b 00000138 ffffffff 00000000
KERNEL32!WaitForSingleObjectEx+0x71
17aced7c 79201b25 00000138 ffffffff 79201b76
KERNEL32!WaitForSingleObject+0xf
17aced88 79201b76 00000001 0ca2ea78 00000000
mscorwks!GCHeap::WaitUntilGCComplete+0x4f
17aced98 792ccbef 0ca270a8 79bb57b4 00100000
mscorwks!Thread::RareDisablePreemptiveGC+0xb5
17acedb0 792ce506 79bb57b4 00100000 17acedd0
mscorwks!gc_heap::mark_through_cards_for_large_objects+0x5ae
17acedc0 791b3c64 00000014 00000002 17acedf0 mscorwks!GCHeap::Alloc+0x12
17acedd0 791d51e1 00000014 00000000 00040000 mscorwks!Alloc+0x3a
17acedf0 7920a2af 79bb57b4 17acee60 17acee3c
mscorwks!FastAllocateObject+0x25
17acee00 7920a309 79bb57b4 0ca2ea78 17acee60
mscorwks!ComObject::CreateComObjectRef+0x23
17acee3c 7920a257 00000000 00000000 00000000
mscorwks!COMInterfaceMarshaler::CreateObjectRef+0x4a
17acee4c 7920910f 17acf610 17acf610 00000000
mscorwks!COMInterfaceMarshaler::FindOrCreateObjectRef+0x43
17acf2ec 7923522a 09485ed8 00000000 00000000
mscorwks!GetObjectRefFromComIP+0x1a3
17acf524 7920c87d 17acf610 7920c846 00000004
mscorwks!InterfaceMarshaler::ConvertSpaceNativeToCom+0x3b
17acf52c 7920c846 00000004 084f0928 00000000
mscorwks!InterfaceMarshaler::UnmarshalComToNativeByrefOut+0x8
17acf5f4 791bfc43 084f094c 17acf690 17acf698 mscorwks!RunML+0x8d9
17acf6bc 00ddb999 0ca2ea78 17acf6f0 0ca0bbb0
mscorwks!ComPlusToComWorker+0x2c2
WARNING: Frame IP not in any known module. Following frames may be wrong.
17acf764 084ec5c7 17f15f08 07e0d849 00000006 0xddb999
17acf7b0 07e0d50f 04f9e394 16e9ed70 16ed8e48 0x84ec5c7
17acf7dc 07e0d4b6 084ec33a 16e9ed70 17f15e5c 0x7e0d50f


Thanks,

Pavel Lebedinsky

unread,
Jul 5, 2003, 2:14:09 AM7/5/03
to
The call stacks show most of the threads waiting for GC
to complete. Thread #25 seems to be starting garbage
collection (it's currently suspending threads).

It would be interesting to see if the process is really deadlocked,
or just unresponsive. Do you know if it's is using any CPU time
when the hang happens?

I would also take a look at .NET CLR Memory perf counters,
it might give some clues as to what's going on.

Finally, you might find the following articles useful:

Production Debugging for .NET Framework Applications
http://msdn.microsoft.com/library/en-us/dnbda/html/DBGch03.asp?frame=true
http://msdn.microsoft.com/library/en-us/dnbda/html/DBGch02.asp?frame=true

Cheung, Jeffrey Jing-Yen

unread,
Jul 6, 2003, 7:42:43 PM7/6/03
to
Pavel,

Thanks for your prompt reply. Yes, it's using CPU time when it hangs; in fact, CPU is at 100% as well before it deadlocks due to the 180 second timeout. Is there some command I
can issue in the debugger to see what the GC is trying to collect, and what exactly is still in memory? Would this provide clues as to what I need to optimize in the code?

Thanks in advance,

Jeff Cheung

Cheung, Jeffrey Jing-Yen

unread,
Jul 7, 2003, 3:44:33 PM7/7/03
to
Ivan,

I've successfully loaded the SOS.dll that MSDN states I need in order to
debug any part of the managed stack, and I've come up with a list of
objects that are awaiting garbage collection. Would GC collection ever
realistically take longer than 3 minutes? If it exceeds this, would it
be possible that the GC is trying to collect something that is rooted,
and subsequently fails? I'm not sure what I'm supposed to do at this
point. I have made sure that all objects are nulled after using them,
so the GC should not have trouble collecting these at all as far as I'm
concerned. Maybe it's something else?

Please advise,

Pavel Lebedinsky

unread,
Jul 7, 2003, 11:16:09 PM7/7/03
to
I don't know much about this. I think sos.dll has some commands
that let you examine managed heap and GC state. You could take
a look at these - may be you'll notice something suspicious, like
way too meany String objects or something like this.

COM interop might be another area to look at. If you don't
explicitly release the wrappers using Marshal.ReleaseComObject
you might consume lots of resources that the GC is not aware of.
Among the things to look for would be high unmanaged memory
usage (such as very large heaps reported by !heap 0), process
handle count, number of DB connections and so on.

You might also try posting your question and all the information you
collected in the microsoft.public.dotnet.framework.aspnet newsgroup.

Ivan Brugiolo [MSFT]

unread,
Jul 8, 2003, 1:26:36 AM7/8/03
to
Unfortunately your problem is getting way too .NET Framework specific,
and I remember problems with garbage collection of "large blocks" in the 1.0
framework,
but I never came across those in your specific context to be able to
recognize them.
I would recomend moving your findings-of-facts in a ASP.NET group, or a
Framewotk group.

--
This posting is provided "AS IS" with no warranties, and confers no rights.
Use of any included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm


"Cheung, Jeffrey Jing-Yen" <jin...@emailREMOVEMEuser.net> wrote in message

news:OCKCwAMR...@TK2MSFTNGP10.phx.gbl...

Cheung, Jeffrey Jing-Yen

unread,
Jul 8, 2003, 11:10:26 AM7/8/03
to
Ivan,

Thanks for all of your help. I will try moving this discussion to an
ASP.NET group, but most of the time they are pretty unresponsive. I
read an article about the 1.0 framework being unable to release large
objects; however, I'm using the 1.1 framework, and I'm guessing this
problem was fixed since then. Again, just want to express my
appreciation for the time spent in helping me out.

Cheung, Jeffrey Jing-Yen

unread,
Jul 8, 2003, 11:17:36 AM7/8/03
to
Pavel,

I think I will try to ask in the aspnet ng, but I do want to thank you
for the information you've provided. I did eventually load the sos.dll
and was able to examine what was in the managed heap. There weren't way
too many String objects, but there was one big HUGE one sitting there
waiting to be collected. In any case, I wanted to thank you for the
invaluable information.

0 new messages