w3wp.exe Hanging - IISState log
Jonathan Steinberg
Our web service methods on our web server use remoting to get data from
components on our app server. This setup has been workgin fine in IIS5.
In our new servers, we are getting a consistent hang of w3ws.exe.
Typically it will have ~60 threads 80M memory at the time.
W3WS.exe is raising this error in the system event log
Application popup: w3wp.exe - Application Error : The instruction at
"0x77bc6f76" referenced memory at "0x00000000". The memory could not be
"read".
Click on OK to terminate the program
Click on CANCEL to debug the program
I am appending the IISstate log for the w3wp process.
(I have clipped some out to fit in th 30k line limit)
Any ideas would be much appreciated
Opened log file 'C:\IISState\output\IISState-1940.log'
***********************
Starting new log output
IISState version 3.3.1
Wed May 25 16:11:15 2005
OS = Windows 2003 Server
Executable: w3wp.exe
PID = 1940
Note: Thread times are formatted as HH:MM:SS.ms
***********************
Thread ID: 0
System Thread ID: 884
Kernel Time: 0:0:0.78
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23
Thread ID: 1
System Thread ID: bf8
Kernel Time: 0:0:0.46
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 2
System Thread ID: 840
Kernel Time: 0:0:0.31
User Time: 0:0:0.15
Thread Type: Other
# ChildEBP RetAddr
00 00a2ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 00a2ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 00a2ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 00a2ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 3
System Thread ID: 930
Kernel Time: 0:0:0.0
User Time: 0:0:0.15
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.
No remote call being made
# ChildEBP RetAddr
00 00aafe18 7c821c54 ntdll!KiFastSystemCallRet
01 00aafe1c 77c7538c ntdll!ZwReplyWaitReceivePortEx+0xc
02 00aaff84 77c5778f RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0x198
03 00aaff8c 77c5f7dd RPCRT4!RecvLotsaCallsWrapper+0xd
04 00aaffac 77c5de88 RPCRT4!BaseCachedThreadRoutine+0x9d
05 00aaffb8 77e66063 RPCRT4!ThreadStartRoutine+0x1b
06 00aaffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 8
System Thread ID: 88c
Kernel Time: 0:0:0.15
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e2fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e2fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e2ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e2ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 9
System Thread ID: 98c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017ffa84 7c822124 ntdll!KiFastSystemCallRet
01 017ffa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017ffaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017ffb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017fffb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017fffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 10
System Thread ID: f1c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Possible ASP page. Possible DCOM activity
Executing Page: ASP.dll symbols not found. Unable to locate ASP page.
Continuing with other analysis.
No remote call being made
# ChildEBP RetAddr
00 0183ff70 7c821364 ntdll!KiFastSystemCallRet
01 0183ff74 77c5fa28 ntdll!NtDelayExecution+0xc
02 0183ff8c 77c5f824 RPCRT4!TIMER::Wait+0x2b
03 0183ffac 77c5de88 RPCRT4!BaseCachedThreadRoutine+0xe8
04 0183ffb8 77e66063 RPCRT4!ThreadStartRoutine+0x1b
05 0183ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 11
System Thread ID: a98
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0187ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0187ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0187ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0187ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 12
System Thread ID: 54c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
succeeded
Loaded Son of Strike data table version 5 from
"C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorwks.dll"
Thread 12
Not a managed thread.
Begin System Thread Information
# ChildEBP RetAddr
00 01a6fe70 7c822114 ntdll!KiFastSystemCallRet
01 01a6fe74 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01a6ff1c 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01a6ff38 791d283b kernel32!WaitForMultipleObjects+0x18
04 01a6ffa0 791d27b2 mscorwks!DebuggerRCThread::MainLoop+0x90
05 01a6ffb0 791d453a mscorwks!DebuggerRCThread::ThreadProc+0x68
06 01a6ffb8 77e66063 mscorwks!DebuggerRCThread::ThreadProcStatic+0xb
07 01a6ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 13
System Thread ID: a18
Kernel Time: 0:0:0.62
User Time: 0:0:0.15
Thread Status: Thread is in a WAIT state.
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 13
ESP EIP
Begin System Thread Information
# ChildEBP RetAddr
00 01b4fdf8 7c822114 ntdll!KiFastSystemCallRet
01 01b4fdfc 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 01b4fea4 77e61075 kernel32!WaitForMultipleObjectsEx+0x11a
03 01b4fec0 792650ff kernel32!WaitForMultipleObjects+0x18
04 01b4fee0 791b9872 mscorwks!WaitForFinalizerEvent+0x5a
05 01b4ff24 791cede0 mscorwks!GCHeap::FinalizerThreadStart+0x96
06 01b4ffb8 77e66063 mscorwks!Thread::intermediateThreadProc+0x44
07 01b4ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 14
System Thread ID: 9c0
Kernel Time: 0:0:0.78
User Time: 0:0:0.15
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 14
ESP EIP
Begin System Thread Information
# ChildEBP RetAddr
00 01ceff74 792ee237 mscorwks!ThreadpoolMgr::DeactivateNthWait+0x1
01 01ceff8c 792ee409 mscorwks!ThreadpoolMgr::ProcessWaitCompletion+0x17
02 01ceffb8 77e66063 mscorwks!ThreadpoolMgr::WaitThreadStart+0x7c
03 01ceffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 15
System Thread ID: d44
Kernel Time: 0:0:0.93
User Time: 0:0:0.93
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 15
Not a managed thread.
Begin System Thread Information
# ChildEBP RetAddr
00 0206feb0 7c821364 ntdll!KiFastSystemCallRet
01 0206feb4 77e42439 ntdll!NtDelayExecution+0xc
02 0206ff1c 77e424b7 kernel32!SleepEx+0x68
03 0206ff2c 792edf9e kernel32!Sleep+0xf
04 0206ffb8 77e66063 mscorwks!ThreadpoolMgr::GateThreadStart+0x54
05 0206ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 16
System Thread ID: 6c8
Kernel Time: 0:0:10.187
User Time: 0:0:4.453
IISState has detected that this thread has thrown an unhandled exception
and is about to crash. IISState will now attempt to display the original
faulting stack.
.cxr 20af744
# ChildEBP RetAddr
00 020afa0c 5a39f9a2 msvcrt!_purecall+0xb
01 020afb50 5a32cf84 w3core!ISAPI_REQUEST::SendResponseHeaders+0x5d
02 020afb78 5a3218b5 w3isapi!SSFSendResponseHeader+0xe0
03 020afbe8 79e76100 w3isapi!ServerSupportFunction+0x351
04 020afc0c 79e7637c aspnet_isapi!HttpCompletion::ReportHttpError+0x3a
05 020afe50 79e7619c
aspnet_isapi!HttpCompletion::ProcessRequestInManagedCode+0x1d1
06 020afe5c 79e8e244 aspnet_isapi!HttpCompletion::ProcessCompletion+0x24
07 020afe70 792ed3e2 aspnet_isapi!CorThreadPoolWorkitemCallback+0x13
08 020afe84 792ed5db mscorwks!ThreadpoolMgr::ExecuteWorkRequest+0x19
09 020afea4 792eccf3 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x129
0a 020affb8 77e66063 mscorwks!ThreadpoolMgr::intermediateThreadProc+0x44
0b 020affec 00000000 kernel32!BaseThreadStart+0x34
**Current Stack State**
Other information: Thread is throwing an exception. Checking for known issues.
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\MSVCR71.dll -
*** WARNING: Unable to verify checksum for D:\PBWeb\Filters\UrlMapFilter.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
D:\PBWeb\Filters\UrlMapFilter.dll -
*** ERROR: Module load completed but symbols could not be loaded for
c:\windows\microsoft.net\framework\v1.1.4322\mscorlib.dll
*** ERROR: Module load completed but symbols could not be loaded for
c:\windows\assembly\gac\system\1.0.5000.0__b77a5c561934e089\system.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\system32\USER32.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\system32\VERSION.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\system32\ADVAPI32.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\system32\msi.dll -
*** WARNING: Unable to verify checksum for
c:\windows\assembly\nativeimages1_v1.1.4322\mscorlib\1.0.5000.0__b77a5c561934e089_eb2d8503\mscorlib.dll
*** ERROR: Module load completed but symbols could not be loaded for
c:\windows\assembly\nativeimages1_v1.1.4322\mscorlib\1.0.5000.0__b77a5c561934e089_eb2d8503\mscorlib.dll
*** WARNING: Unable to verify checksum for
c:\windows\microsoft.net\framework\v1.1.4322\temporary asp.net
files\root\58d42327\990f262b\assembly\dl2\870d2a06\00b60a49_b24ac501\webqueries.dll
*** ERROR: Module load completed but symbols could not be loaded for
c:\windows\microsoft.net\framework\v1.1.4322\temporary asp.net
files\root\58d42327\990f262b\assembly\dl2\870d2a06\00b60a49_b24ac501\webqueries.dll
Thread Type: Other
# ChildEBP RetAddr
00 020af3a4 7c821b74 ntdll!KiFastSystemCallRet
01 020af3a8 77e999ea ntdll!NtRaiseHardError+0xc
02 020af614 77e840a1 kernel32!UnhandledExceptionFilter+0x4b4
03 020af61c 77e6b7f9 kernel32!BaseThreadStart+0x4a
04 020af644 7c82eeb2 kernel32!_except_handler3+0x61
05 020af668 7c82ee84 ntdll!ExecuteHandler2+0x26
06 020af710 7c82ecc6 ntdll!ExecuteHandler+0x24
07 020af730 77bc6f76 ntdll!KiUserExceptionDispatcher+0xe
08 020af7f8 77bc6f76 msvcrt!_purecall+0xb
Thread ID: 17
System Thread ID: 5b0
Kernel Time: 0:0:0.31
User Time: 0:0:0.15
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 17
Not a managed thread.
Begin System Thread Information
# ChildEBP RetAddr
00 0428ff38 7c821364 ntdll!KiFastSystemCallRet
01 0428ff3c 77e42439 ntdll!NtDelayExecution+0xc
02 0428ffa4 792ee51d kernel32!SleepEx+0x68
03 0428ffb8 77e66063 mscorwks!ThreadpoolMgr::TimerThreadStart+0x30
04 0428ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 21
System Thread ID: af4
Kernel Time: 0:0:0.15
User Time: 0:0:0.15
Thread Status: Thread is in a WAIT state.
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 21
ESP EIP
00aefaf4 7c82ed54 [FRAME: ECallMethodFrame] [DEFAULT] Boolean
System.Threading.WaitHandle.WaitOneNative(I,UI4,Boolean)
00aefb08 799e4bb1 [DEFAULT] [hasThis] Boolean
System.Threading.WaitHandle.WaitOne(I4,Boolean)
00aefb3c 01f9ae60 [DEFAULT] Void
System.EnterpriseServices.ServicedComponentProxy.QueueCleaner()
00aefd44 7923c069 [FRAME: GCFrame]
Begin System Thread Information
# ChildEBP RetAddr
00 00aef94c 7c822114 ntdll!KiFastSystemCallRet
01 00aef950 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 00aef9f8 7929a834 kernel32!WaitForMultipleObjectsEx+0x11a
03 00aefa28 7929ad95 mscorwks!Thread::DoAppropriateWaitWorker+0xc1
04 00aefa7c 79339ff6 mscorwks!Thread::DoAppropriateWait+0x46
05 00aefac4 01e74df0 mscorwks!WaitHandleNative::CorWaitOneNative+0x6f
WARNING: Frame IP not in any known module. Following frames may be wrong.
06 00aefb30 01f9ae60 0x1e74df0
07 00aefb40 7923e0bc 0x1f9ae60
08 00aefc50 7923e2a7 mscorwks!MethodDesc::CallDescr+0x1b8
09 00aefd0c 7923e315 mscorwks!MethodDesc::CallDescr+0x4f
0a 00aefd34 792e9207 mscorwks!MethodDesc::Call+0x97
0b 00aefd80 792e92d6 mscorwks!ThreadNative::KickOffThread_Worker+0x9d
0c 00aefe24 791cede0 mscorwks!ThreadNative::KickOffThread+0xc2
0d 00aeffb8 77e66063 mscorwks!Thread::intermediateThreadProc+0x44
0e 00aeffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 22
System Thread ID: 8fc
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 22
ESP EIP
0598fa24 7c82ed54 [FRAME: ECallMethodFrame] [DEFAULT] Boolean
System.Threading.WaitHandle.WaitOneNative(I,UI4,Boolean)
0598fa38 799e4bb1 [DEFAULT] [hasThis] Boolean
System.Threading.WaitHandle.WaitOne(I4,Boolean)
0598fa6c 04be654f [DEFAULT] Void
System.EnterpriseServices.ServicedComponentProxy.QueueCleaner()
0598fc74 7923c069 [FRAME: GCFrame]
0598fcbc 7923c069 [FRAME: ContextTransitionFrame]
Begin System Thread Information
# ChildEBP RetAddr
00 0598f87c 7c822114 ntdll!KiFastSystemCallRet
01 0598f880 77e6711b ntdll!NtWaitForMultipleObjects+0xc
02 0598f928 7929a834 kernel32!WaitForMultipleObjectsEx+0x11a
03 0598f958 7929ad95 mscorwks!Thread::DoAppropriateWaitWorker+0xc1
04 0598f9ac 79339ff6 mscorwks!Thread::DoAppropriateWait+0x46
05 0598f9f4 01e74df0 mscorwks!WaitHandleNative::CorWaitOneNative+0x6f
WARNING: Frame IP not in any known module. Following frames may be wrong.
06 0598fa60 04be654f 0x1e74df0
07 0598fa70 7923e0bc 0x4be654f
08 0598fb80 7923e2a7 mscorwks!MethodDesc::CallDescr+0x1b8
09 0598fc3c 7923e315 mscorwks!MethodDesc::CallDescr+0x4f
0a 0598fc64 792e9207 mscorwks!MethodDesc::Call+0x97
0b 0598fcb0 7929cc30 mscorwks!ThreadNative::KickOffThread_Worker+0x9d
0c 0598fcf8 792e92cf mscorwks!Thread::DoADCallBack+0x5c
0d 0598fda4 791cede0 mscorwks!ThreadNative::KickOffThread+0xbb
0e 0598ffb8 77e66063 mscorwks!Thread::intermediateThreadProc+0x44
0f 0598ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 23
System Thread ID: cc8
Kernel Time: 0:0:0.906
User Time: 0:0:0.765
*** ERROR: Module load completed but symbols could not be loaded for
c:\windows\microsoft.net\framework\v1.1.4322\temporary asp.net
files\root\58d42327\990f262b\assembly\dl2\a39eb4bd\802dd7ff_5dc0c401\pdflib_dotnet.dll
IISState has detected that this thread has thrown an unhandled exception
and is about to crash. IISState will now attempt to display the original
faulting stack.
.cxr 6d7ed44
# ChildEBP RetAddr
00 06d7f00c 5a39f9a2 msvcrt!_purecall+0xb
01 06d7f150 5a32cf84 w3core!ISAPI_REQUEST::SendResponseHeaders+0x5d
02 06d7f178 5a3218b5 w3isapi!SSFSendResponseHeader+0xe0
03 06d7f1e8 79e76100 w3isapi!ServerSupportFunction+0x351
04 06d7f20c 79e7637c aspnet_isapi!HttpCompletion::ReportHttpError+0x3a
05 06d7f450 79e7619c
aspnet_isapi!HttpCompletion::ProcessRequestInManagedCode+0x1d1
06 06d7f45c 79e8e244 aspnet_isapi!HttpCompletion::ProcessCompletion+0x24
07 06d7f470 792ed3e2 aspnet_isapi!CorThreadPoolWorkitemCallback+0x13
08 06d7f484 792ed5db mscorwks!ThreadpoolMgr::ExecuteWorkRequest+0x19
09 06d7f4a4 792eccf3 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x129
0a 06d7ffb8 77e66063 mscorwks!ThreadpoolMgr::intermediateThreadProc+0x44
0b 06d7ffec 00000000 kernel32!BaseThreadStart+0x34
**Current Stack State**
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 23
ESP EIP
Begin System Thread Information
# ChildEBP RetAddr
00 06d7e664 7c821b74 ntdll!KiFastSystemCallRet
01 06d7e668 77e999ea ntdll!NtRaiseHardError+0xc
02 06d7e8d4 01001623 kernel32!UnhandledExceptionFilter+0x4b4
03 06d7e8f0 7c35f0c3 w3wp!WpUnhandledExceptionFilter+0x3d
WARNING: Stack unwind information not available. Following frames may be
wrong.
04 06d7e9a0 792b0e99 MSVCR71!_unDNameEx+0x764
05 06d7e9b0 0466ae68 mscorwks!COMUnhandledExceptionFilter+0xd
06 06d7ec14 77e840a1 pdflib_dotnet+0xaae68
07 06d7ec1c 77e6b7f9 kernel32!BaseThreadStart+0x4a
08 06d7ec44 7c82eeb2 kernel32!_except_handler3+0x61
09 06d7ec68 7c82ee84 ntdll!ExecuteHandler2+0x26
0a 06d7ed10 7c82ecc6 ntdll!ExecuteHandler+0x24
0b 06d7ed30 77bc6f76 ntdll!KiUserExceptionDispatcher+0xe
0c 06d7edf8 77bc6f76 msvcrt!_purecall+0xb
Thread ID: 24
System Thread ID: d30
Kernel Time: 0:0:0.93
User Time: 0:0:0.15
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 01aaff24 7c821bf4 ntdll!KiFastSystemCallRet
01 01aaff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 01aaff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 01aaff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 01aaffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 01aaffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 01aaffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 25
System Thread ID: 644
Kernel Time: 0:0:0.46
User Time: 0:0:0.62
IISState has detected that this thread has thrown an unhandled exception
and is about to crash. IISState will now attempt to display the original
faulting stack.
.cxr 440f2c4
# ChildEBP RetAddr
00 0440f58c 5a39f9a2 msvcrt!_purecall+0xb
01 0440f6d0 5a32cf84 w3core!ISAPI_REQUEST::SendResponseHeaders+0x5d
02 0440f6f8 5a3218b5 w3isapi!SSFSendResponseHeader+0xe0
03 0440f768 79e76100 w3isapi!ServerSupportFunction+0x351
04 0440f78c 79e7637c aspnet_isapi!HttpCompletion::ReportHttpError+0x3a
05 0440f9d0 79e7619c
aspnet_isapi!HttpCompletion::ProcessRequestInManagedCode+0x1d1
06 0440f9dc 79e8e244 aspnet_isapi!HttpCompletion::ProcessCompletion+0x24
07 0440f9f0 792ed3e2 aspnet_isapi!CorThreadPoolWorkitemCallback+0x13
08 0440fa04 792ed5db mscorwks!ThreadpoolMgr::ExecuteWorkRequest+0x19
09 0440fa24 792eccf3 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x129
0a 0440ffb8 77e66063 mscorwks!ThreadpoolMgr::intermediateThreadProc+0x44
0b 0440ffec 00000000 kernel32!BaseThreadStart+0x34
**Current Stack State**
Other information: Thread is throwing an exception. Checking for known issues.
*** WARNING: Unable to verify checksum for
c:\windows\microsoft.net\framework\v1.1.4322\temporary asp.net
files\root\58d42327\990f262b\assembly\dl2\cf7ecd76\00d7125b_f850c501\portalui.dll
*** ERROR: Module load completed but symbols could not be loaded for
c:\windows\microsoft.net\framework\v1.1.4322\temporary asp.net
files\root\58d42327\990f262b\assembly\dl2\cf7ecd76\00d7125b_f850c501\portalui.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\system32\OLEAUT32.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorsn.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\system32\mscoree.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\system32\cryptnet.dll -
Thread Type: Other
# ChildEBP RetAddr
00 0440ef24 7c821b74 ntdll!KiFastSystemCallRet
01 0440ef28 77e999ea ntdll!NtRaiseHardError+0xc
02 0440f194 77e840a1 kernel32!UnhandledExceptionFilter+0x4b4
03 0440f19c 77e6b7f9 kernel32!BaseThreadStart+0x4a
04 0440f1c4 7c82eeb2 kernel32!_except_handler3+0x61
05 0440f1e8 7c82ee84 ntdll!ExecuteHandler2+0x26
06 0440f290 7c82ecc6 ntdll!ExecuteHandler+0x24
07 0440f2b0 77bc6f76 ntdll!KiUserExceptionDispatcher+0xe
08 0440f378 77bc6f76 msvcrt!_purecall+0xb
Thread ID: 54
System Thread ID: f54
Kernel Time: 0:0:0.62
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
Thread 54
ESP EIP
Begin System Thread Information
# ChildEBP RetAddr
00 09b8f778 7c822124 ntdll!KiFastSystemCallRet
01 09b8f77c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 09b8f7ec 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 09b8f800 792ed4ec kernel32!WaitForSingleObject+0x12
04 09b8f824 792eccf3 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x3a
05 09b8ffb8 77e66063 mscorwks!ThreadpoolMgr::intermediateThreadProc+0x44
06 09b8ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 55
System Thread ID: 484
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0990ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 0990ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 0990ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 0990ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 56
System Thread ID: 940
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 09a4ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 09a4ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 09a4ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 09a4ffec 00000000 kernel32!BaseThreadStart+0x34
Thread ID: 57
System Thread ID: a2c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0960ff70 7c821bf4 ntdll!KiFastSystemCallRet
01 0960ff74 7c83ad75 ntdll!NtRemoveIoCompletion+0xc
02 0960ffb8 77e66063 ntdll!RtlpWorkerThread+0x3d
03 0960ffec 00000000 kernel32!BaseThreadStart+0x34
*****SUMMARY*****
--
Jonathan Steinberg
Banc of America Securities
Prime Brokerage Services
Pat [MSFT]
button to be clicked). You have 3 threads that are actually in the process
of crashing, which is a very hard thing to do.
They are all in the process of handling ISAPI functions, so I would say that
you are having a problem w/an ISAPI filter.
I would suspect this one:
D:\PBWeb\Filters\UrlMapFilter.dll
But it is very hard to tell b/c basically the bad thing happened, then a
bunch of cleanup work was done, then the bad thing became unrecoverable and
you crashed.
I would suggest running IISState in 'soft crash mode' to see if we can catch
the culprit early:
iisstate -p <pid of w3wp.exe> -sc <enter>
then wait for the crash
Then post the log.
Pat
"Jonathan Steinberg" <Jonathan...@discussions.microsoft.com> wrote in
message news:3649BE5E-1DE4-4824...@microsoft.com...
Jonathan Steinberg
I did as you suggested, and in the process I have identified which
application is causing the crashes.
We have a app which is repreating a method call every 5 seconds if it does
not receive a good response. This is the method which is causing the w3wp
crash. After a certain number of crashes, the whole process hangs.
I am currently investigating why this method fails, but I have to ask why
this problem would cause the entire w3wp process to hang. Is there a way to
configure IIS6 so the application pop-up from w3wp does not hang the server,
or is this a bug?
--
Jonathan Steinberg
Pat [MSFT]
Basically a pop-up message box telling you that a crash occurred is blocking
all of the threads. So, if you eliminate the crash, you eliminate the
pop-up.
Pat
"Jonathan Steinberg" <Jonathan...@discussions.microsoft.com> wrote in
message news:8C5E78D1-B1F3-4025...@microsoft.com...
Jonathan Steinberg
I know that the process is crashing. My question is why does one
application's method calls crash the entire website.
I was asking if there is a was to have the server handle my application's
errors without crashing.
Pat [MSFT]
application (in IIS terms) is just a logical construct for various
structures (e.g. application variables), it does not create a unique process
unless you run it in High isolation. In that case application = process.
Pat
"Jonathan Steinberg" <Jonathan...@discussions.microsoft.com> wrote in
message news:965B8E9D-F652-4A7E...@microsoft.com...
Jonathan Steinberg
A call to MS Premier support led to the answer to this.
The w3wp.exe crash is caused by a bug in ISAPI in Windows 2003.
This bug appears when you implement an IHttpAsyncHandler handler and
set the completed state in the AsyncResult in BeginProcessRequest :
Sample crashing code
class Async:IAsyncResult
{
/*Snipped IAsyncResult code
public bool CompletedSynchronously
{
get
{return false;}
}
private Boolean isCompleted;
public bool IsCompleted
{
get
{
return isCompleted;
}
}
internal void SetCompleted()
{
isCompleted = true;
if(cb != null)
{
cb(this);
}
}
// state internal fields
internal HttpContext context=null;
}
public class AsyncHandler : IHttpAsyncHandler
{
public IAsyncResult BeginProcessRequest(
HttpContext context, AsyncCallback cb, Object extraData)
{
Async async = new Async(cb, extraData);
async.context = context;
/* THIS IS THE PROBLEM CODE */
async.SetCompleted();
// return IAsyncResult object to caller
return async;
}
}
On Win2k, this code works fine. On wind2003, it causes an AV. The problem is
related to the CompletedSynchronously method of the asyncresult.
If the completion flag of the asyncresult, is set in BeginProcessRequest,
CompletedSynchronously must return true.
The workaround for this is to rewrite the asyncresult class like this:
class Async:IAsyncResult
{
/*Snipped IAsyncResult code
privte bool completedSynch
public bool CompletedSynchronously
{
get
{return completedSynch;}
}
private Boolean isCompleted;
public bool IsCompleted
{
get
{
return isCompleted;
}
}
internal void SetCompleted(bool synch)
{
isCompleted = true;
compeltedSynch = synch;
if(cb != null)
{
cb(this);
}
}
// state internal fields
internal HttpContext context=null;
}
The code in BeginProcessRequest then becomes :
async.SetCompleted(true);
This solves the crash, as the async result CompletedSynchronously method
will now return true.