Re: Issue 108 in data-race-test: Support Windows x64

[data-ra...@googlecode.com]

Updates:
Summary: Support Windows x64
Labels: -Type-Defect Type-Enhancement OpSys-Windows

Comment #1 on issue 108 by timu...@google.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

Windows x64 is not supported by TSan(v1) at this moment.
Technically, there are no known roadblocks; we've just never aimed at Win
x64 support and don't plan to as of today.

One day I got some Win x64-related patches, and I already committed the
most obvious bits into TSan sources.
Unfortunately, I've never heard back from the author.
That said, your patches are welcome!

I'd optimistically estimate the effort as "a few days to read the existing
and understand the Windows bits" and "a week to fix all problems in the
x64-specific parts".
One of the reasons we don't work on this is that we don't have a decent
multi-threaded Win x64 app to test TSan on.
It should be easier for you guys as you do have such an app.
If you can provide new minimized test cases for racecheck_unittest.cc that
show some problems on Win x64 I might help you with some advice on how to
fix them.

---
Also, there's TSan v2 in active development now:
http://code.google.com/p/thread-sanitizer/
It should be doable to add support for Win_x64 as soon as Clang works there
(hm, that's not close now) and with some effort to port the runtime.

[data-ra...@googlecode.com]

Comment #2 on issue 108 by jhu...@gmail.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

That sounds great - I'd like to pursue this. We definitely have a good test
app - about 500K lines of heavily multi-threaded C++ (closed source). I
downloaded the source, but the build process appears to be all
Linux-oriented. I am familiar with makefiles etc. on Linux though. Are
there instructions somewhere for the 32-bit Windows build?

[data-ra...@googlecode.com]

Comment #3 on issue 108 by timu...@google.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

Hm, I'm pretty sure we had a separate wiki page for that, not sure why it
was deleted [was out of date?]

See how it's built on our buildbot:
http://build.chromium.org/p/client.tsan/builders/buildbot-win7
-> pick the latest build
-> hit on the "stdio" link just below "build tsan with pin" or "build debug
tsan with pin"
-> look for the "argv" line at the top.

Please note that we're currently using an slightly out-of-date version of
PIN which has bugs on Win x64, you should probably rebase the TSan sources
to work with a newer PIN.

[data-ra...@googlecode.com]

Comment #4 on issue 108 by timu...@google.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

See
http://code.google.com/p/data-race-test/source/diff?spec=svn3607&old=3606&r=3607&format=side&path=%2Fwiki%2FThreadSanitizerPin.wiki
for the old version of the instruction.

[data-ra...@googlecode.com]

Comment #5 on issue 108 by timu...@google.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

Ah, sorry - it was a Linux PIN page.

If you need help besides c#3, please ping me.
Otherwise, it'd be good if you could send me some text to add to a Windows
subsection of
http://code.google.com/p/data-race-test/wiki/BuildingThreadSanitizer

[data-ra...@googlecode.com]

Comment #6 on issue 108 by jhu...@gmail.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

Okay - thanks Timur. I should have time to get started on that today. I'll
let you know if I run into any issues.

Jan

[data-ra...@googlecode.com]

Comment #7 on issue 108 by jhu...@gmail.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

I installed Cygwin and I'm trying the 32-bit make, but it isn't finding
pin.H. I wonder if it's because the cl command is using /I instead of -I.
Here's the make command and output, with an "ls" at the bottom to show that
pin.H is really there.

$ make -C tsan -j4 VALGRIND_ROOT= PIN_ROOT=/cygdrive/e/pin w32d
make: Entering directory `/cygdrive/e/tsanv1/tsan'
make all OS=windows ARCH=x86 DEBUG=1
make[1]: Entering directory `/cygdrive/e/tsanv1/tsan'
VALGRIND_ROOT is not set. Not building the Valgrind-based variant.
cl -nologo -c -Zi -MT -EHs- -EHa- -wd4530 -D_CRT_SECURE_NO_DEPRECATE
-D_SECURE_SCL=0 -D_HAS_ITERATOR_DEBUGGING=0 -Gy -Ox -GL
/I/cygdrive/e/pin/source/include /I/cygdrive/e/pin/source/include/gen
/I/cygdrive/e/pin/extras/xed2-ia32/include /DTARGET_WINDOWS
/DBIGARRAY_MULTIPLIER=1 /DUSING_XED /DTARGET_IA32 /DHOST_IA32a
/D_SECURE_SCL=0 -DTS_PIN=1 -Fobin/x86-windows-debug-pinmp-ts_pin.obj -c
ts_pin.cc -DTS_VERSION=\"0\" -DDEBUG=1 -I../dynamic_annotations
-DTS_SERIALIZED=0
ts_pin.cc
ts_pin.cc(32) : fatal error C1083: Cannot open include file: 'pin.H': No
such file or directory
Makefile:253: recipe for target `bin/x86-windows-debug-pinmp-ts_pin.obj'
failed
make[1]: *** [bin/x86-windows-debug-pinmp-ts_pin.obj] Error 2
make[1]: Leaving directory `/cygdrive/e/tsanv1/tsan'
Makefile:145: recipe for target `w32d' failed
make: *** [w32d] Error 2
make: Leaving directory `/cygdrive/e/tsanv1/tsan'

$ ls /cygdrive/e/pin/source/include
compiler_version_check2.H level_core.PLH pin.H
pin_profile.H pin-errtype.h
foundation.PLH level_foundation.PLH pin_basic.h
pin_ucontext.h pintool.exp
gen level_pinclient.PLH pin_cache.H
pin_util.H pintool.ver
level_base.PLH level_vmapi.PLH pin_isa.H
pinapp.h portability.H

[data-ra...@googlecode.com]

Comment #8 on issue 108 by jhu...@gmail.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

I tried entering the cl command by hand with -I but that didn't work either:

$ cl -nologo -c -Zi -MT -EHs- -EHa- -wd4530 -D_CRT_SECURE_NO_DEPRECATE

-D_SECURE_SCL=0 -D_HAS_ITERATOR_DEBUGGING=0 -Gy -Ox -GL

-I/cygdrive/e/pin/source/include -I/cygdrive/e/pin/source/include/gen
-I/cygdrive/e/pin/extras/xed2-ia32/include -DTARGET_WINDOWS
-DBIGARRAY_MULTIPLIER=1 -DUSING_XED -DTARGET_IA32 -DHOST_IA32a
-D_SECURE_SCL=0 -DTS_PIN=1 -Fobin/x86-windows-debug-pinmp-ts_pin.obj -c

[data-ra...@googlecode.com]

Comment #9 on issue 108 by timu...@google.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

[FTR the problem has been solved over e-mail by replacing "-I/cygdrive/e/"
with "-IE:/" ]

[data-ra...@googlecode.com]

Comment #10 on issue 108 by wolen...@chromium.org: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

Is Win64 support in TSAN v1 still being actively worked on?

--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[data-ra...@googlecode.com]

Comment #11 on issue 108 by konstant...@gmail.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

Not that I know of. We also mostly abandoned tsan1 on other platforms and
switched to tsan2 (code.google.com/p/thread-sanitizer/)

[data-ra...@googlecode.com]

Comment #12 on issue 108 by jhu...@gmail.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

I was able to get tsan1 building on Win64 and I got the unit tests to pass,
but there are a lot of problems running it (false positives in the Windows
runtime mainly), and I don't have the time or expertise to fix that. When
will tsan2 be ready to try?

[data-ra...@googlecode.com]

Comment #13 on issue 108 by dvyu...@google.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

The main question is -- do you able to build your codebase with clang/gcc?
Porting of tsan2 runtime to windows should not be a rocket science. But we
do not invest in it ATM because we are unable to build our codebase with
clang/gcc on windows.

[data-ra...@googlecode.com]

Comment #14 on issue 108 by jhu...@gmail.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

Right - so tsan2 doesn't apply to most Windows apps, such as mine.
Hopefully somebody will maintain tsan1 then, and help get the Win64 support
working!

[data-ra...@googlecode.com]

Comment #15 on issue 108 by timu...@google.com: Support Windows x64
http://code.google.com/p/data-race-test/issues/detail?id=108

RE: codegen on Windows
I'm currently working on this for ASan for Win x86 (see issue
address-sanitizer:56) and once we're there it shouldn't be too hard to add
x64 support and TSan v2 runtime.

However, I wouldn't expect quick results (the task is huge), so Windows is
tsanv1-only for now.