Analyzing a loader lock-related deadlock
Tim De Baets
application. One of the users reported a hang when the application starts up,
and there are no other programs on his system that show the same behavior. I
asked him to send me the crash dump, which revealed that the hang occurs in a
3rd party dll (scuredll.dll). I suggested the user to rename this dll and
reboot, but my application still hangs, now in MSCTFIME.IME, so it's becoming
very unlikely that the dll reported in WinDbg is the real culprit.
The fact that my application hangs at the loader lock really surprises me
because it doesn't depend on any custom dll's, it's practically one single
exe that can run by itself on any Windows system. The hang could be caused by
a 3rd party dll on the user's system, but this would be weird too as it only
happens in my application and not any other.
Could someone please take a look at this and try to figure out what exactly
is going wrong, or give me some useful pointers?
WinDbg's output is below, the full crash dump can be downloaded from
http://users.skynet.be/tim.debaets/SmartClose.exe.hdmp
Thanks
Tim De Baets
--
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(770.d4): Application hang - code cfffffff (first/second chance not available)
eax=00000206 ebx=00000000 ecx=001d0098 edx=000a0006 esi=7c97c0d8 edi=00000000
eip=7c90eb94 esp=0012ed58 ebp=0012ede0 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=d669 es=48ca fs=003b gs=ffff efl=00000246
ntdll!KiFastSystemCallRet:
7c90eb94 c3 ret
0:000> !analyze -v
*******************************************************************************
*
*
* Exception Analysis
*
*
*
*******************************************************************************
Unable to load image C:\Program Files\SmartClose\SmartClose.exe, Win32 error
0n2
*** WARNING: Unable to verify timestamp for SmartClose.exe
*** ERROR: Module load completed but symbols could not be loaded for
SmartClose.exe
Matched: 7c8866fc kernel32!BasepAppCertDllsList = <no type information>
Matched: 7c8848d0 kernel32!BasepAppCertDllsList = <no type information>
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: kernel32!pNlsUserInfo ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: kernel32!pNlsUserInfo ***
*** ***
*************************************************************************
FAULTING_IP:
ntdll!KiFastSystemCallRet+0
7c90eb94 c3 ret
EXCEPTION_RECORD: ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 7c90eb94 (ntdll!KiFastSystemCallRet)
ExceptionCode: cfffffff (Application hang)
ExceptionFlags: 00000000
NumberParameters: 0
BUGCHECK_STR: HANG
PROCESS_NAME: SmartClose.exe
ERROR_CODE: (NTSTATUS) 0xcfffffff - <Unable to get error code text>
NTGLOBALFLAG: 0
APPLICATION_VERIFIER_FLAGS: 0
CRITICAL_SECTION: 7c97c0d8 -- (!cs -s 7c97c0d8)
BLOCKING_THREAD: 00000da8
LOADERLOCK_BLOCKED_API:
GetModuleHandleForUnicodeString:LdrGetDllHandle:LdrGetDllHandleEx:LdrLockLoaderLock:
LOADERLOCK_OWNER_API:
_LdrpInitialize:LdrpInitializeThread:LdrpCallInitRoutine:
DERIVED_WAIT_CHAIN:
Dl Eid Cid WaitType
-- --- ------- --------------------------
0 770.d4 Critical Section -->
1 770.da8 Unknown
WAIT_CHAIN_COMMAND: ~0s;k;;~1s;k;;
DEFAULT_BUCKET_ID: APPLICATION_HANG_BusyHang
PRIMARY_PROBLEM_CLASS: APPLICATION_HANG_BusyHang
LAST_CONTROL_TRANSFER: from 755da01e to 755d90a6
FAULTING_THREAD: 00000001
STACK_TEXT:
0123fbf0 755da01e 755c0000 00000002 00000000 MSCTFIME!DllMain+0x41
0123fc10 7c9011a7 755c0000 00000002 00000000 MSCTFIME!_DllMainCRTStartup+0x52
0123fc30 7c918f65 755d9fcc 755c0000 00000002 ntdll!LdrpCallInitRoutine+0x14
0123fca4 7c918dde 0123fd30 0123fd30 0000000c ntdll!LdrpInitializeThread+0xc0
0123fd1c 7c90eac7 0123fd30 7c900000 00000000 ntdll!_LdrpInitialize+0x219
00000000 00000000 00000000 00000000 00000000 ntdll!KiUserApcDispatcher+0x7
FOLLOWUP_IP:
MSCTFIME!DllMain+41
755d90a6 5e pop esi
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: MSCTFIME!DllMain+41
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: MSCTFIME
IMAGE_NAME: MSCTFIME.IME
DEBUG_FLR_IMAGE_TIMESTAMP: 411096eb
STACK_COMMAND: ~1s ; kb
BUCKET_ID: HANG_MSCTFIME!DllMain+41
FAILURE_BUCKET_ID: MSCTFIME.IME!DllMain_cfffffff_APPLICATION_HANG_BusyHang
Followup: MachineOwner
---------
0:000> !locks
CritSec ntdll!LdrpLoaderLock+0 at 7c97c0d8
LockCount 1
RecursionCount 1
OwningThread da8
EntryCount 1
ContentionCount 1
*** Locked
Scanned 214 critical sections
0:000> !cs -s 7c97c0d8
-----------------------------------------
Critical section = 0x7c97c0d8 (ntdll!LdrpLoaderLock+0x0)
DebugInfo = 0x7c97c100
LOCKED
LockCount = 0x1
OwningThread = 0x00000da8
RecursionCount = 0x1
LockSemaphore = 0x94
SpinCount = 0x00000000
ntdll!RtlpStackTraceDataBase is NULL. Probably the stack traces are not
enabled.
Tim De Baets
myself a while ago. The application started hanging on my own pc as
well, and when I looked at it in Process Explorer I saw that one of the
threads was suspended. Manually resuming the thread using Process
Explorer brought my application back to life, so it looks like the
suspended thread was holding the loader lock and causing the deadlock.
Further investigation showed that a Delphi component for displaying GIF
images was responsible for this suspended thread. More specifically, the
TTimerEx component that is used by TGIFImage from sulako.com. The
author, Theodor Kleynhans, has been informed of this bug, but hasn't
send any response so far.
I have now switched to another component for displaying GIF images and
haven't experienced any deadlocks since then. Users of TGIFImage are
hereby warned. This also shows that 3rd party DLLs doing unholy things
in their DllMain aren't the only possible cause of loader lock-related
deadlocks: misbehaving 3rd party components can be the culprit too.
Regards
Tim De Baets