Trouble with Windows ASan build

[Ken Rockot]

'ello good people of the chromium. I'm failing at this.

Just FYI, GYP flags in use: clang=1 asan=1 component=static_library disable_nacl=1

I am trying to figure out if I'm doing something wrong or if depot_tools is broken with Windows + clang.

gclient runhooks dies with this output:

Running ['svn', 'checkout', '--force', 'https://llvm.org/svn/llvm-project/compiler-rt/trunk@217738', 'D:\\src\\chrome\\src\\third_party\\llvm\\projects\\compiler-rt'] (try #1)

Traceback (most recent call last):

  File "src/tools/clang/scripts/update.py", line 249, in <module>

  File "src/tools/clang/scripts/update.py", line 245, in main

  File "src/tools/clang/scripts/update.py", line 169, in UpdateClang

  File "D:\src\chrome\src\tools\gyp\pylib\gyp\MSVSVersion.py", line 106, in SetupScript

    os.path.join(self.path, 'VC/vcvarsall.bat')), arg]

  File "D:\src\chrome\depot_tools\python276_bin\lib\ntpath.py", line 96, in join

    assert len(path) > 0

TypeError: object of type 'NoneType' has no len()

Error: Command D:\src\chrome\depot_tools\python276_bin\python.exe src/tools/clang/scripts/update.py --if-needed returned

 non-zero exit status 1 in D:\src\chrome

Hook ''D:\src\chrome\depot_tools\python276_bin\python.exe' src/tools/clang/scripts/update.py --if-needed' took 99.82 secs

It looks like the MSVSVersion code makes assumptions in several places that it's OK for path to be None, yet there are several calls to os.path.join which assume path is not None. 

Though even if it were not None in this case, it looks like there is also an assumption that VC/vcvarsall.bat should exist. This also appears to be wrong.

Pls halp, wat do!

[Victor Khimenko]

On Sat, Sep 20, 2014 at 1:40 AM, Ken Rockot <roc...@chromium.org> wrote:

'ello good people of the chromium. I'm failing at this.

Just FYI, GYP flags in use: clang=1 asan=1 component=static_library disable_nacl=1

I am trying to figure out if I'm doing something wrong or if depot_tools is broken with Windows + clang.

Do you actually have MSVC installed? Clang is not yet advanced enough to compile everything in Chromium thus you still need MSVC to build the rest. Messages below just look like you don't have MSVC installed properly:

  http://www.chromium.org/developers/how-tos/build-instructions-windows#TOC-Setting-up-the-environment-for-Visual-Studio-2013

 

gclient runhooks dies with this output:

Running ['svn', 'checkout', '--force', 'https://llvm.org/svn/llvm-project/compiler-rt/trunk@217738', 'D:\\src\\chrome\\src\\third_party\\llvm\\projects\\compiler-rt'] (try #1)

Traceback (most recent call last):

  File "src/tools/clang/scripts/update.py", line 249, in <module>

  File "src/tools/clang/scripts/update.py", line 245, in main

  File "src/tools/clang/scripts/update.py", line 169, in UpdateClang

  File "D:\src\chrome\src\tools\gyp\pylib\gyp\MSVSVersion.py", line 106, in SetupScript

    os.path.join(self.path, 'VC/vcvarsall.bat')), arg]

  File "D:\src\chrome\depot_tools\python276_bin\lib\ntpath.py", line 96, in join

    assert len(path) > 0

TypeError: object of type 'NoneType' has no len()

Error: Command D:\src\chrome\depot_tools\python276_bin\python.exe src/tools/clang/scripts/update.py --if-needed returned

 non-zero exit status 1 in D:\src\chrome

Hook ''D:\src\chrome\depot_tools\python276_bin\python.exe' src/tools/clang/scripts/update.py --if-needed' took 99.82 secs

It looks like the MSVSVersion code makes assumptions in several places that it's OK for path to be None, yet there are several calls to os.path.join which assume path is not None. 

Though even if it were not None in this case, it looks like there is also an assumption that VC/vcvarsall.bat should exist. This also appears to be wrong.

Pls halp, wat do!

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Scott Graham]

I don't think there's a prebuilt binary for clang-win yet, so it requires a system-installed VS and CMake to build itself.

You also might try finding a corresponding bot on the waterfall or chromium.fyi to compare settings.

[Scott Graham]

(Oh, and Win Clang Asan and Win SyzyAsan aren't the same thing, in case you happen to be trying to repro a bug that SyzyAsan turned up.)

[Nico Weber]

https://code.google.com/p/chromium/wiki/Clang#Windows has some documentation. Note that win asan isn't considered to be ready for general use, see the list of missing bits on that page. (See Scott's note about Win Asan and Win SyzyAsan though, the latter's supposed to work.)

[Ken Rockot]

Ah OK. So I need a real system install of MSVS. I was assuming the stuff in depot_tools/win_toolchain should have been sufficient.

I am trying to repro a Clang Asan build, not a SyzyAsan.

Actually now that I think about it, I do have a system install of 2012 Express - is Express not sufficient here?

[Scott Graham]

2012 won't work. If that was a typo, and you have 2013 Express, it might be enough to set GYP_MSVS_VERSION=2013e.

[Timur Iskhodzhanov]

[/sub]

Ken,
Yes, I think the problem is that depot_tools has failed to find a
proper VS on the PATH.

Scott,
Do you think depot_tools error message could be a bit more descriptive?

--
Tim

[Scott Graham]

On Mon, Sep 22, 2014 at 1:38 AM, Timur Iskhodzhanov <timu...@chromium.org> wrote:

[/sub]

Ken,
Yes, I think the problem is that depot_tools has failed to find a
proper VS on the PATH.

Scott,
Do you think depot_tools error message could be a bit more descriptive?

I think the error will be earlier for "normal" builds, only tools/clang/scripts/update.py pokes inside gyp. But... yes: https://codereview.chromium.org/595543003

[Hans Wennborg]

Apologies for not seeing this thread earlier.

On Fri, Sep 19, 2014 at 3:39 PM, Ken Rockot <roc...@chromium.org> wrote:

> Ah OK. So I need a real system install of MSVS. I was assuming the stuff in
> depot_tools/win_toolchain should have been sufficient.

update.py piggybacks on gyp to find the MSVS installation. Having a
"real" system install should not be necessary; it should be using the
one in depot_tools/win_toolchain.

The script always requests the "2013" version. Maybe it needs to also
try "2013e" for users who have the express version. Can you check what
version you have in depot_tools/win_toolchain?

Thanks,
Hans

[Hans Wennborg]

On Mon, Sep 22, 2014 at 9:47 AM, Hans Wennborg <ha...@chromium.org> wrote:
> Apologies for not seeing this thread earlier.
>
> On Fri, Sep 19, 2014 at 3:39 PM, Ken Rockot <roc...@chromium.org> wrote:
>> Ah OK. So I need a real system install of MSVS. I was assuming the stuff in
>> depot_tools/win_toolchain should have been sufficient.
>
> update.py piggybacks on gyp to find the MSVS installation. Having a
> "real" system install should not be necessary; it should be using the
> one in depot_tools/win_toolchain.

Hmm, actually I see now that on our buildbots it's using the system
MSVS - not the one under depot_tools like I thought it would :/ I'll
see if I can fix this.

[Scott Graham]

On Mon, Sep 22, 2014 at 9:59 AM, Hans Wennborg <ha...@chromium.org> wrote:

On Mon, Sep 22, 2014 at 9:47 AM, Hans Wennborg <ha...@chromium.org> wrote:
> Apologies for not seeing this thread earlier.
>
> On Fri, Sep 19, 2014 at 3:39 PM, Ken Rockot <roc...@chromium.org> wrote:
>> Ah OK. So I need a real system install of MSVS. I was assuming the stuff in
>> depot_tools/win_toolchain should have been sufficient.
>
> update.py piggybacks on gyp to find the MSVS installation. Having a
> "real" system install should not be necessary; it should be using the
> one in depot_tools/win_toolchain.

Hmm, actually I see now that on our buildbots it's using the system
MSVS - not the one under depot_tools like I thought it would :/ I'll
see if I can fix this.

It would need to use the code in build/vs_toolchain.py to set up the correct environment.

[Nico Weber]

(FWIW, I don't think we should spend much time making this experience nicer. The real fix is to get things ready enough to have prebuilt clang binaries on Windows, and spending time on making the current thing nicer slows us down in getting there.)

[Hans Wennborg]

This should be fixed now in
https://chromium.googlesource.com/chromium/src/+/28fb41c40e75fa0896058996d7cde447902b3ed2