Questions about use of blacklists with Dr. Memory

Jim Monte

unread,

Dec 3, 2019, 1:35:40 PM12/3/19

to Dr. Memory Users

Hi All,

In Windows, I am trying to exclude some errors by adding a -lib_blacklist flag when Dr. Memory is run. Here is what the current documentation claims at https://drmemory.org/docs/page_options.html

-lib_blacklist <string>
default: ""
Error reports whose top N frames' module paths match any of these ,-separated patterns will be separated by default as merely potential errors, where N is -lib_blacklist_frames. These errors are reported to potential_errors.txt rather than results.txt. This feature is disabled if -lib_blacklist_frames is 0. The -lib_whitelist takes priority over this blacklist: i.e., if any top frame matches the whitelist, the error will be reported normally, even if all frames also match the blacklist. Each pattern can use * and ? wildcards (which have the same semantics as in suppression files) and is matched against the full path of each module. The default on Windows is set to $SYSTEMROOT*.d?? if not otherwise specified.

Without the flag, the summary in the results.txt file is

ERRORS FOUND:

0 unique, 0 total unaddressable access(es)

17 unique, 99 total uninitialized access(es)

0 unique, 0 total invalid heap argument(s)

0 unique, 0 total GDI usage error(s)

0 unique, 0 total handle leak(s)

0 unique, 0 total warning(s)

0 unique, 0 total, 0 byte(s) of leak(s)

0 unique, 0 total, 0 byte(s) of possible leak(s)

ERRORS IGNORED:

151 potential error(s) (suspected false positives)

(details: d:\software\ngspice\work\testing\paranoia\DrMemory-ngspice.exe.9100.000\potential_errors.txt)

80 potential leak(s) (suspected false positives)

(details: d:\software\ngspice\work\testing\paranoia\DrMemory-ngspice.exe.9100.000\potential_errors.txt)

915 unique, 5944 total, 1321820 byte(s) of still-reachable allocation(s)

(re-run with "-show_reachable" for details)

Adding -lib_blacklist $SYSTEMROOT*.d??, which is supposed to be the default, and running as before, the results become

ERRORS FOUND:

0 unique, 0 total unaddressable access(es)

76 unique, 311 total uninitialized access(es)

0 unique, 0 total invalid heap argument(s)

0 unique, 0 total GDI usage error(s)

0 unique, 0 total handle leak(s)

0 unique, 0 total warning(s)

36 unique, 62 total, 23670 byte(s) of leak(s)

41 unique, 290 total, 19500 byte(s) of possible leak(s)

ERRORS IGNORED:

92 potential error(s) (suspected false positives)

(details: d:\software\ngspice\work\testing\paranoia\DrMemory-ngspice.exe.36400.000\potential_errors.txt)

915 unique, 5944 total, 1321820 byte(s) of still-reachable allocation(s)

(re-run with "-show_reachable" for details)

Quoting the string, so that it becomes -lib_blacklist "$SYSTEMROOT*.d??", and again running the as for the first time, the results are

ERRORS FOUND:

0 unique, 0 total unaddressable access(es)

76 unique, 326 total uninitialized access(es)

0 unique, 0 total invalid heap argument(s)

0 unique, 0 total GDI usage error(s)

0 unique, 0 total handle leak(s)

0 unique, 0 total warning(s)

36 unique, 62 total, 23670 byte(s) of leak(s)

44 unique, 290 total, 19500 byte(s) of possible leak(s)

ERRORS IGNORED:

92 potential error(s) (suspected false positives)

(details: d:\software\ngspice\work\testing\paranoia\DrMemory-ngspice.exe.23844.000\potential_errors.txt)

914 unique, 5943 total, 1317648 byte(s) of still-reachable allocation(s)

(re-run with "-show_reachable" for details)

What I would like to do for the current test is to add USER32.dll to the defaults, but I am unable to even get the defaults again when I explicitly add the -lib_blacklist flag with the defaults. Besides a -logdir flag, the -lib_blacklist flag was the only one I added. Any help would be appreciated!

Jim

Derek Bruening

unread,

Dec 4, 2019, 1:06:08 AM12/4/19

to drmemor...@googlegroups.com

$SYSTEMROOT is meant to be the value of that environment variable, rather than the literal string "$SYSTEMROOT": the option does not expand embedded environment variables.

It looks like Microsoft Shared is also added by default but the docs fail to mention that. You can see that here: https://github.com/DynamoRIO/drmemory/blob/master/drmemory/frontend.c#L1425

--

---
You received this message because you are subscribed to the Google Groups "Dr. Memory Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drmemory-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/drmemory-users/55a4110d-abd1-4e41-a2f2-77b956e0b9a5%40googlegroups.com.

Jim Monte

unread,

Dec 4, 2019, 11:40:12 AM12/4/19

to Dr. Memory Users

Thank you! The link to the source code doing the work was helpful. I now can get back to the default behavior. However, looking at the expanded list, I am not understanding why the errors involving user32.dll were even showing up using the defaults. Below is my full command.

d:\software\ngspice\work\testing\paranoia\examples\control_structs>drmemory -logdir ..\.. -lib_blacklist "C:\WINDOWS*.d??,C:\Program Files\Common Files\microsoft shared*.d??,C:\Program Files (x86)\Common Files\microsoft shared*.d??,C:\Windows\System32\user32.dll,USER32.dll" -- D:\software\ngspice\source\ngspice_rw\visualc\vngspice\Debug.x64\ngspice.exe -o ..\..\s-param.log s-param.cir

When I start the ngspice.exe in Visual Studio, it shows the module C:\Windows\System32\user32.dll and no other user32.dll DLL among the loaded modules. As I understand blacklists, the first one in the list, C:\WINDOWS*.d?? should have blocked it. At the end of the blacklist, I explicitly put user32.dll both with the full path, and when that did not work. with only the name of the DLL, since that is how Dr. Memory reported it. However, I am still getting errors like

Error #2: UNINITIALIZED READ: reading 0x00000000007ff7f4-0x00000000007ff7f8 4 byte(s) within 0x00000000007ff7f0-0x00000000007ff7f8

# 0 system call NtUserCreateWindowEx parameter value #14

# 1 USER32.dll!CreateWindowExW +0x9f0 (0x00007ffeaa8ecee1 <USER32.dll+0xcee1>)

# 2 USER32.dll!CreateWindowExW +0x247 (0x00007ffeaa8ec738 <USER32.dll+0xc738>)

# 3 USER32.dll!CreateWindowExA +0x81 (0x00007ffeaa909f02 <USER32.dll+0x29f02>)

# 4 WinMain [D:\software\ngspice\source\ngspice_rw\src\winmain.c:948]

Note: @0:00:01.568 in thread 1912

Error #3: UNINITIALIZED READ: reading 0x00000000007ff7f4-0x00000000007ff7f8 4 byte(s) within 0x00000000007ff7f0-0x00000000007ff7f8

# 0 system call NtUserCreateWindowEx parameter value #14

# 1 USER32.dll!CreateWindowExW +0x9f0 (0x00007ffeaa8ecee1 <USER32.dll+0xcee1>)

# 2 USER32.dll!CreateWindowExW +0x247 (0x00007ffeaa8ec738 <USER32.dll+0xc738>)

# 3 USER32.dll!CreateWindowExA +0x81 (0x00007ffeaa909f02 <USER32.dll+0x29f02>)

# 4 WinMain [D:\software\ngspice\source\ngspice_rw\src\winmain.c:959]

Note: @0:00:02.611 in thread 1912

Error #4: UNINITIALIZED READ: reading 0x00000000007ff7f4-0x00000000007ff7f8 4 byte(s) within 0x00000000007ff7f0-0x00000000007ff7f8

# 0 system call NtUserCreateWindowEx parameter value #14

# 1 USER32.dll!CreateWindowExW +0x9f0 (0x00007ffeaa8ecee1 <USER32.dll+0xcee1>)

# 2 USER32.dll!CreateWindowExW +0x247 (0x00007ffeaa8ec738 <USER32.dll+0xc738>)

# 3 USER32.dll!CreateWindowExA +0x81 (0x00007ffeaa909f02 <USER32.dll+0x29f02>)

# 4 WinMain [D:\software\ngspice\source\ngspice_rw\src\winmain.c:985]

Note: @0:00:03.899 in thread 1912

Error #5: UNINITIALIZED READ: reading 0x00000000007ff7f4-0x00000000007ff7f8 4 byte(s) within 0x00000000007ff7f0-0x00000000007ff7f8

# 0 system call NtUserCreateWindowEx parameter value #14

# 1 USER32.dll!CreateWindowExW +0x9f0 (0x00007ffeaa8ecee1 <USER32.dll+0xcee1>)

# 2 USER32.dll!CreateWindowExW +0x247 (0x00007ffeaa8ec738 <USER32.dll+0xc738>)

# 3 USER32.dll!CreateWindowExA +0x81 (0x00007ffeaa909f02 <USER32.dll+0x29f02>)

# 4 WinMain [D:\software\ngspice\source\ngspice_rw\src\winmain.c:1005]

Note: @0:00:03.925 in thread 1912

What am I doing wrong?

Jim

To unsubscribe from this group and stop receiving emails from it, send an email to drmemor...@googlegroups.com.

Derek Bruening

unread,

Dec 4, 2019, 11:44:37 AM12/4/19

to drmemor...@googlegroups.com

-lib_blacklist_frames is 4 by default, requiring 4 user32 frames in a row. Setting -lib_blacklist_frames to 3 (or smaller) should match those callstacks.

To unsubscribe from this group and stop receiving emails from it, send an email to drmemory-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/drmemory-users/6ae448d3-5762-40a1-8775-8a12fb4a0092%40googlegroups.com.

Jim Monte

unread,

Dec 4, 2019, 1:06:37 PM12/4/19

to Dr. Memory Users

Thanks again. As I understood the documentation at https://drmemory.org/docs/page_options.html, I thought this option selected the number of frames starting at the top that would be checked for a blacklisted item and that those errors would be excluded, but using -lib_blacklist_frames 3 did what I wanted, almost.

I still had one error from kernel32.dll that I wanted suppressed.

Error #1: UNINITIALIZED READ: reading register eax

# 0 __strncnt [minkernel\crts\ucrt\src\appcrt\string\strncnt.cpp:21]

# 1 __acrt_LCMapStringA_stat [minkernel\crts\ucrt\src\appcrt\locale\lcmapstringa.cpp:58]

# 2 __acrt_LCMapStringA [minkernel\crts\ucrt\src\appcrt\locale\lcmapstringa.cpp:224]

# 3 setSBUpLow [minkernel\crts\ucrt\src\appcrt\mbstring\mbctype.cpp:634]

# 4 _setmbcp_nolock [minkernel\crts\ucrt\src\appcrt\mbstring\mbctype.cpp:837]

# 5 setmbcp_internal [minkernel\crts\ucrt\src\appcrt\mbstring\mbctype.cpp:424]

# 6 __acrt_initialize_multibyte [minkernel\crts\ucrt\src\appcrt\mbstring\mbctype.cpp:920]

# 7 __acrt_execute_initializers [minkernel\crts\ucrt\src\appcrt\internal\shared_initialization.cpp:25]

# 8 __acrt_initialize [minkernel\crts\ucrt\src\appcrt\internal\initialization.cpp:287]

# 9 __scrt_initialize_crt [d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\utility\utility.cpp:199]

#10 __scrt_common_main_seh [d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:237]

#11 __scrt_common_main [d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:330]

#12 WinMainCRTStartup [d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_winmain.cpp:16]

#13 KERNEL32.dll!BaseThreadInitThunk +0x13 (0x00007ffead288364 <KERNEL32.dll+0x8364>)

Note: @0:00:00.608 in thread 4132

Note: instruction: test %eax %eax

Since it only appeared once, I used -lib_blacklist_frames 1. That had no effect on kernel32.dll, although I again verified that its path was among the blacklisted ones.

While I could not get rid of that last error, the list of errors was close enough to the ones I wanted to consider for me to add the -pause_at_uninitialized flag and attach a debugger to see details on the program state when one of the errors occurred. That did not work as expected. The call stack for the executing thread at the time and the call stack for the thread causing the uninitialized read identified by Dr. Memory (and the call stacks for any of the other threads) were nothing like the one reported by Dr. Memory. Continuing execution or stepping maybe a few dozen instructions caused an access violation. Without Dr. Memory at all or with it but without the -pause_at_uninitialized flag, there have been no issues with access violations. What needs to be done when attaching a debugger so the call stack at the time of the error can be seen?

To view this discussion on the web visit https://groups.google.com/d/msgid/drmemory-users/6ae448d3-5762-40a1-8775-8a12fb4a0092%40googlegroups.com.

2019-12-04 12_47_32_main-vngspice.png

2019-12-04 12_43_15_at_break-.png

Derek Bruening

unread,

Dec 5, 2019, 12:49:55 PM12/5/19

to drmemor...@googlegroups.com

On Wed, Dec 4, 2019 at 1:06 PM Jim Monte <jim.m...@gmail.com> wrote:

Thanks again. As I understood the documentation at https://drmemory.org/docs/page_options.html, I thought this option selected the number of frames starting at the top that would be checked for a blacklisted item and that those errors would be excluded, but using -lib_blacklist_frames 3 did what I wanted, almost.

I still had one error from kernel32.dll that I wanted suppressed.

Error #1: UNINITIALIZED READ: reading register eax
# 0 __strncnt [minkernel\crts\ucrt\src\appcrt\string\strncnt.cpp:21]
# 1 __acrt_LCMapStringA_stat [minkernel\crts\ucrt\src\appcrt\locale\lcmapstringa.cpp:58]
# 2 __acrt_LCMapStringA [minkernel\crts\ucrt\src\appcrt\locale\lcmapstringa.cpp:224]
# 3 setSBUpLow [minkernel\crts\ucrt\src\appcrt\mbstring\mbctype.cpp:634]
# 4 _setmbcp_nolock [minkernel\crts\ucrt\src\appcrt\mbstring\mbctype.cpp:837]
# 5 setmbcp_internal [minkernel\crts\ucrt\src\appcrt\mbstring\mbctype.cpp:424]
# 6 __acrt_initialize_multibyte [minkernel\crts\ucrt\src\appcrt\mbstring\mbctype.cpp:920]
# 7 __acrt_execute_initializers [minkernel\crts\ucrt\src\appcrt\internal\shared_initialization.cpp:25]
# 8 __acrt_initialize [minkernel\crts\ucrt\src\appcrt\internal\initialization.cpp:287]
# 9 __scrt_initialize_crt [d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\utility\utility.cpp:199]
#10 __scrt_common_main_seh [d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:237]
#11 __scrt_common_main [d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:330]
#12 WinMainCRTStartup [d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_winmain.cpp:16]
#13 KERNEL32.dll!BaseThreadInitThunk +0x13 (0x00007ffead288364 <KERNEL32.dll+0x8364>)
Note: @0:00:00.608 in thread 4132
Note: instruction: test %eax %eax

Since it only appeared once, I used -lib_blacklist_frames 1. That had no effect on kernel32.dll, although I again verified that its path was among the blacklisted ones.

As the documentation says, the -lib_blacklist only looks at frames at the top of the callstack. So it would not be expected to do anything with this callstack. You could make a general pattern in a suppression file to suppress anything with kernel32 anywhere in it. However, I would not advise that, nor would I advise suppressing this callstack based on it having kernel32 in it, because the kernel32 here is in *every* callstack that makes it to the bottom: that's generic thread init code that is always run and you would end up suppressing real errors. This particular error is https://github.com/DynamoRIO/drmemory/issues/2237 which was added to the default suppressions in https://github.com/DynamoRIO/drmemory/commit/fb7f04622dcd435ae02a76a492bd363d704377d1.

To unsubscribe from this group and stop receiving emails from it, send an email to drmemory-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/drmemory-users/5d21d28b-6279-4596-b38b-9256052e8ed9%40googlegroups.com.

Derek Bruening

unread,

Dec 5, 2019, 1:22:29 PM12/5/19

to drmemor...@googlegroups.com

On Wed, Dec 4, 2019 at 1:06 PM Jim Monte <jim.m...@gmail.com> wrote:

While I could not get rid of that last error, the list of errors was close enough to the ones I wanted to consider for me to add the -pause_at_uninitialized flag and attach a debugger to see details on the program state when one of the errors occurred. That did not work as expected. The call stack for the executing thread at the time and the call stack for the thread causing the uninitialized read identified by Dr. Memory (and the call stacks for any of the other threads) were nothing like the one reported by Dr. Memory.

We have long wanted to provide transparent debugging support (https://github.com/DynamoRIO/drmemory/issues/600) but have not had the engineer time to implement it. If someone would like to help develop that feature the contribution would be appreciated.

For now the current PC and top few frames are inside the tool when attaching at the pause point. To switch to the application context requires a few steps, most easily done in windbg. FIrst, load the symbols using the load_syms64 script provided by DynamoRIO. Then, use the CONTEXT local variable provided by Dr. Memory in the drmemorylib!report_pause_at_error frame to swap to the app context via the windbg command .cxr. This would look like this from the windbg prompt:

$><c:\downloaded\path\to\load_syms64
~0s
kn
.frame 4
.cxr cxt
kn

We will try to add this to the documentation (better would be to implement issue 600 linked above but help is likely needed for that....)

Continuing execution or stepping maybe a few dozen instructions caused an access violation.

Note that Dr. Memory uses page faults to implement efficient shadow memory operations, so just seeing an access violation in the debugger does not mean something is wrong. Just continue past expected access violations.

To unsubscribe from this group and stop receiving emails from it, send an email to drmemory-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/drmemory-users/5d21d28b-6279-4596-b38b-9256052e8ed9%40googlegroups.com.

Reply all

Reply to author

Forward