Verifier.dll

[Nikita Desjardins]

While debugging the crash on customer's machine we noticed that our application runs under verifier.dll. e.g each thread stack starts with verifier.dll and every system function is intercepted with verifier.dll stub.

Common sense and running Google search for "verifier.dll" clearly indicates that this dll belongs to Application Verifier tool provided by Microsoft. However the stunning fact is that Application Verifier not even installed on customer's machine (appverif.exe not found on client's machine neither c:\Program Files\Application Verifier directory),.

Application Verifier is a debugging feature of the operating system, so it is always there. It can be configured by modifying the registry even if you haven't got the Application Verifier tool installed on the system.

This article on MSDN gives details of the registry settings that control the debugging features of the operating system. It might be that someone or something has enabled application verifier on the system that you are seeing the exception on. If it isn't specifically enabled for your process it might have been enabled system wide.

Application Verifier is a runtime verification tool used to find bugs in Microsoft Windows-applications. Since it is a runtime tool the application code needs to be exercised in order to be verified. Good test coverage is therefore essential.

The typical usage scenario of Application Verifier is to enable it for the applications of interest (see questions below for how to do that) and then run all the tests you have written for your application. You will get a notification for any bug found in the form of a debugger break or a verifier log entry.

After installing Application Verifier you can either start it by accessing it in your list of programs OR by typing Appverif.exe on a command line. To do this, go to a command prompt or the Run box of the Startup menu. Type appverif.exe and then press Enter. This will start Application Verifier.

AppVerifier cares about the interfaces between the operating system and the application. As a result, unless your managed code is performing interop against native APIs that have to do with Heaps, Handles, Critical Section, etc. your test cases are not going to give you any interaction with the interfaces that are verified.

The Basics verification layer within Application Verifier require that you run your application under a debugger. If you do not have a debugger associated to the application prior to selecting the test, you will receive a dialog reminding you that you will need to run your application under a debugger in order to obtain the logged information.

In general, stack expansion should be really tested in isolation from other verification layers, including heap. The reason is the following: each verification layers "thunks" an API or an exported point with some routine.

For example, a call to CreateFileA, will be a call to appvocre!NS_SecurityChecks::CreateFileA, that might call appvcore!NS_FillePaths::CreateFileA that might call kernel32!CreateFileA, that might call verifier!AVrfpNtCreateFile, that will call ntdll!NtCreateFile. You can see that the instrumentation has added 3 more "stacked" function calls, each one of them may and will consume more stack.

In the case below, the LH-verifier.dll is "thunking" every DllMain, and the "instrumented" heap code path will add more stack usage. Since the injected thread from the debugger does not use the IMAGE_NT_HEADERS defaults, the initially committed stack will not be enough to complete the APC state of a thread (a thread in the APC state executed the initialization code).

Simple version: The golden rule for enabling verifier settings on a given application is to match the bit-ness of the tool and the target process. That is: use the 32-bit appverif.exe for a 32-bit application (both running under WoW64) and use the 64-bit AppVerif.exe for the native 64-bit native target.

The "Debugger" value is read from the launching application. So, if you want to have 32-bit devenv.exe launching 64-bit my.exe and have it running under debugger, you must use the 32-bit registry key under WoW6432Node. The other values, for a 32-bit process, are read from both places, both the native IFEO and the WoW6432Node.

The reasoning is the following: a 32-bit process running under WoW is a 64-bit process running the Wow64 emulation loop. So, each 32-bit process is first a 64-bit process, and then a 32-bit process. The 64-bit IFEO will enable verifier on the Wow64cpu.dll code, while the 32-bit IFEO will enable verifier on the 32-bit code.

From the end-user point of view, verifier.dll is loaded twice (once in the 64-bit world, once in the 32-bit world). Since most of the people do not care about verifying wow64cpu.dll, the most accepted behavior for 32-bit processes is to only verify the 32-bit part. That is why the golden rule of "always match the bit-ness" applies.

Does AppVerifier do leak detection?Under Windows 7 and greater, there is a Leaks check option that will detect when a process leaks memory. Under earlier operating systems, AppVerifier does not test application for leak detection but looks for other memory issues.

Keep in mind that ObsoleteAPICalls will just spit out a warning for every call it sees to an API that is listed as obsolete or deprecated in MSDN. You should decide on a case by case basis if it is important for your application to switch to the new APIs. Some of the APIs are dangerous, and some have merely been superseded by a newer API with more options. Take a look at the "Dangerous APIs" section of Writing Secure Code, 2nd addition for more.

For applications that need to be highly reliable, like services and server programs, you should also enable the Stacks check. This checks to see if the stack commit size is adequate, by disabling stack growth. If the application quits immediately with a stack overflow, that means the application needs to be recompiled with a larger stack commit size. If you're a tester, and encounter a problem with an application while using the Stacks check, file a bug, assign it to your developer, and keep on testing.

If your process is supposed to stay alive a long time, then these leaks can bite you. Since the fixes are very easy in 99% of the cases (developer just forgot to call RtlDeleteCriticalSection) you should address them.

Establishing an exception handler in the initial thread function is not guaranteed to catch the potential stack overflows that might be raised. This is because the code that dispatches exceptions needs also a little bit of stack to execute on top of the current activation record. Since we just failed the stack extension it is very likely that we will step over the end of the committed stack and raise a second exception while trying to dispatch the first one. A double fault exception will terminate the process unconditionally.

You need to destroy the window before you get the process-detach. The danger is not that user32 will be unloaded. The danger is that you are being unloaded. So the next message that the window receives will crash the process because user32 will deliver the message to your Wndproc which does not exist any more.

Microsoft Windows operating system has thread affinity. Process-detach does not. The loader lock is not really the big problem; the problem is Dllmain. Process-detach is the last time your DLL gets to run code. You must get rid of everything before you return. But since Windows has thread affinity, you cannot clean up the window if you are on the wrong thread.

The loader lock enters into the picture if somebody has a global hook installed (e.g., spy++ is running). In this case, you enter a potential deadlock scenario. Again, the solution is to destroy the window before you get process-detach.

When you commit stack you are just reserving page file space. There is no performance impact. No physical memory is actually used. The only additional cost happens if you actually touch the stack space that you committed. But this will happen anyway even if you do not commit the stack upfront.

There are interesting items, such as exception dispatching on IA64/AMD64 that requires "unexpected" extra stack. There might be some processing happening on RPC worker threads whose stack requirements are past reasonable attempts to measure them.

First of all, you should get an idea of all the thread pools living in the process. The NT-Thread-Pool, with the alertable-wait-threads is sometimes special, because, for example, if you use a database component from SQL, it will use alertable sleeps over a thread that is a target of user-APC. This can cause problems with nested calls.

Once you know all the thread pools, get an idea of how to control their stack requirements. For Example, RPC reads a registry key for the stack commit. The WDM pump threads get that from the image. For Other Thread Pools, the mileage may vary.

When you all threads are clear you can take some action. Not having a huge reserved space helps address space fragmentation only if threads comes and goes very often. If you have a stable thread pool that is in your control, then you might have an advantage in reducing the reserved space as well. It will really help in saving address space for the heaps and address space for users.

This condition creates a camouflage for a more devious attack. An administrator scanning for an attack activity in progress will see nothing wrong with the person requesting ALL_ACCESS on key X, because a particular application always does that. The administrator will think "the person is probably just running Word". The administrator cannot tell that a hacker has penetrated my account and is now probing the system to determine what access I have, which he can exploit for his nefarious ends. The possibilities are endless.

The ACL issue with ALL_ACCESS is that you must always be granted it. If we wanted to someday deny you DELETE access to a certain key, we would not be able to. Even though you were not actually deleting the key, we would be breaking your application because you would request delete access.

c80f0f1006