What is the correct usage for the -msan flag?

[Markus Zimmermann]

Hi gophers!

I am trying to integrate -msan for testing into https://github.com/go-clang/bootstrap using Go 1.7.1 and LLVM 3.9.0 inside an Ubuntu Trusty 64bit VM. The Kernel is 3.13.0.

I am executing a test using the following command:

CC=clang CGO_LDFLAGS="-L`llvm-config --libdir` -fsanitize=memory" CGO_CPPFLAGS='-fsanitize=memory -fsanitize-memory-track-origins -fno-omit-frame-pointer' go test -timeout 60s -v -msan -test.run=TestBasicParsing

And the address sanitizer detects a problem for this test giving me the following output:

Uninitialized bytes in __interceptor_memchr at offset 6 inside [0x70800000bf58, 7)
==32156==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x7fdbcc7e3c85  (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x653c85)
    #1 0x7fdbcc7f40a7  (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x6640a7)
    #2 0x7fdbd0723bbe  (/usr/lib/x86_64-linux-gnu/libclang-3.9.so.1+0xc05bbe)
    #3 0x7fdbd07334ca  (/usr/lib/x86_64-linux-gnu/libclang-3.9.so.1+0xc154ca)
    #4 0x7fdbcffaef39  (/usr/lib/x86_64-linux-gnu/libclang-3.9.so.1+0x490f39)
    #5 0x7fdbcff8c504  (/usr/lib/x86_64-linux-gnu/libclang-3.9.so.1+0x46e504)
    #6 0x7fdbcfd8178e  (/usr/lib/x86_64-linux-gnu/libclang-3.9.so.1+0x26378e)
    #7 0x7fdbcc7be314  (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x62e314)
    #8 0x7fdbcc7be393  (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x62e393)
    #9 0x7fdbcc8214bc  (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x6914bc)
    #10 0x7fdbcf908183  (/lib/x86_64-linux-gnu/libpthread.so.0+0x8183)
    #11 0x7fdbced0d37c  (/lib/x86_64-linux-gnu/libc.so.6+0xfa37c)

  Uninitialized value was stored to memory at
    #0 0x42d0d5  (/tmp/go-build816629300/github.com/go-clang/bootstrap/clang/_test/clang.test+0x42d0d5)
    #1 0x7fdbcbf479bd  (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xbb9bd)

  Uninitialized value was stored to memory at
    #0 0x42d0d5  (/tmp/go-build816629300/github.com/go-clang/bootstrap/clang/_test/clang.test+0x42d0d5)
    #1 0x7fdbcbf46e2f  (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xbae2f)

  Uninitialized value was created by a heap allocation
    #0 0x42b6d9  (/tmp/go-build816629300/github.com/go-clang/bootstrap/clang/_test/clang.test+0x42b6d9)
    #1 0x7fdbcbeeadac  (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x5edac)

SUMMARY: MemorySanitizer: use-of-uninitialized-value (/usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1+0x653c85)

I have the following questions concerning the usage of the -msan flag:
a.) Are these the preferred arguments to use the -msan flag?
b.) Why are there no line numbers in the traces?
c.) Any advice on how I can debug such problems?
d.) What are the official minimum requirements for the -msan flag? (LLVM >= 3.8? ...?)

I appreciate any input since I am completely lost on how to address this problem :-/

Cheers,
Markus

[Ian Lance Taylor]

Looks OK to me.

> b.) Why are there no line numbers in the traces?

I don't know. Try adding -g to CGO_CPPFLAGS to see if it makes a difference.

> c.) Any advice on how I can debug such problems?

If -g doesn't help then I think you'll have to use objdump or gdb or
lldb to look at the code at the failing address. Those tools should
at least let you map that address back to a source line. Note that
the problem does not seem to be in Go code, which of course is
unsurprising.

> d.) What are the official minimum requirements for the -msan flag? (LLVM >=
> 3.8? ...?)

Yes, on GNU/Linux you need at least LLVM 3.8. I don't know of any
other requirements.

Ian

[Markus Zimmermann]

On Friday, September 9, 2016 at 1:51:49 PM UTC+2, Ian Lance Taylor wrote:

> I have the following questions concerning the usage of the -msan flag:
> a.) Are these the preferred arguments to use the -msan flag?

Looks OK to me.

I am wondering if they are really needed, and if yes, why are they not included at compile time by the Go compiler? Also, would it make sense to add -fsanitize-memory-track-origins as -msan-track-origins flag to Go build?

> b.) Why are there no line numbers in the traces?

I don't know.  Try adding -g to CGO_CPPFLAGS to see if it makes a difference.

> c.) Any advice on how I can debug such problems?

If -g doesn't help then I think you'll have to use objdump or gdb or
lldb to look at the code at the failing address.  Those tools should
at least let you map that address back to a source line.  Note that
the problem does not seem to be in Go code, which of course is
unsurprising.

Unfortunately -g does not make a difference. However, you are right that this is not a Go problem and I will create a bug report in the LLVM tracker.

> d.) What are the official minimum requirements for the -msan flag? (LLVM >=
> 3.8? ...?)

Yes, on GNU/Linux you need at least LLVM 3.8.  I don't know of any
other requirements.

I am wondering if the version requirement should be added to the help text of -msan? It currently states "enable interoperation with memory sanitizer. Supported only on linux/amd64, and only with Clang/LLVM as the host C compiler." I am also wondering why these conditions (as stated in https://golang.org/misc/cgo/testsanitizers/test.bash ) are not checked during compile time? Would that be a good Go addition?

Thanks for your answers. They are really helpful.

Cheers,
Markus

[Ian Lance Taylor]

On Tue, Sep 13, 2016 at 5:03 AM, Markus Zimmermann <zim...@gmail.com> wrote:
> On Friday, September 9, 2016 at 1:51:49 PM UTC+2, Ian Lance Taylor wrote:
>>
>> > I have the following questions concerning the usage of the -msan flag:
>> > a.) Are these the preferred arguments to use the -msan flag?
>>
>> Looks OK to me.
>
> I am wondering if they are really needed, and if yes, why are they not
> included at compile time by the Go compiler?

The Go tool will add the -fsanitize=memory option, so you shouldn't
need to add that yourself. It won't add the other options, though.

> Also, would it make sense to
> add -fsanitize-memory-track-origins as -msan-track-origins flag to Go build?

I think not, because I think that is a choice for the user to make.

>> > d.) What are the official minimum requirements for the -msan flag? (LLVM
>> > >=
>> > 3.8? ...?)
>>
>> Yes, on GNU/Linux you need at least LLVM 3.8. I don't know of any
>> other requirements.
>
> I am wondering if the version requirement should be added to the help text
> of -msan? It currently states "enable interoperation with memory sanitizer.
> Supported only on linux/amd64, and only with Clang/LLVM as the host C
> compiler." I am also wondering why these conditions (as stated in
> https://golang.org/misc/cgo/testsanitizers/test.bash ) are not checked
> during compile time? Would that be a good Go addition?

I don't think it makes sense to add specific version numbers to the
-msan help text. The specific details can change with changing Linux
versions, and really the compatibility has nothing to do with Go.
Perhaps it would make sense to say something like "requires a version
of LLVM that supports -fsanitize=memory on the version of GNU/Linux
that the program runs on." I don't know how much that would help
people, though.

Ian