LNK1000 with linkopts "-Wl,<anything>.lib"

[yury.p...@gmail.com]

Hi

Linking a shared library under Windows that uses external libs (e.g. "-Wl,ws2_32.lib") causes link.exe to crash with a message:

fatal error LNK1000: Internal error during CImplib::EmitThunk

The funny think is that building .exe is perfectly ok. Only the combination of "-shared" and "-Wl,some.lib" makes MS linker to fail.

There is certainly a workaround using #pragma comment lib, but would be great to have "-Wl" working as well. I am not quite sure that it is a bazel problem but it is somehow related. Here is a demo project for this issue: https://github.com/yprokazov/buzel

Has anyone faced same trouble? Should I make a ticket at GitHub issue tracker?

Thanks,

Yury

[Yun Peng]

Hi Yury,

I had the same error when building the shared library (a dll) in TensorFlow with GPU support, the library I needed to link is cudart.lib.

The reason why it's failing is basically a bug in MSVC linker.

When building a shared library, Bazel enables the whole archive feature, which means telling the linker to force link every library the target depends on.

On windows, if your VS is or newer than VS 2015 update 2, it means adding /WHOLEARCHIVE option(a new feature in MSVC) in the linking command. And the linker fails with this error when trying to force link system lib files.

pcloudy@PCLOUDY1-W MSYS ~/workspace/buzel

$ cat ./bazel-out/vc_14_0_x64-opt/bin/link_crasher.exe-2.params.msvc

/nologo

/MACHINE:X64

/SUBSYSTEM:CONSOLE

/OUT:bazel-out/vc_14_0_x64-opt/bin/link_crasher.exe

/WHOLEARCHIVE

/DLL

ws2_32.lib

/MT

C:\temp\_bazel_pcloudy\k-8MzLG1\execroot\buzel\bazel-out\vc_14_0_x64-opt\bin\_objs\link_crasher\one.o

(The actual link command would be link @bazel-out/vc_14_0_x64-opt/bin/link_crasher.exe-2.params.msvc)

For VS older than 2015 update 2, /WHOLEARCHIVE doesn't exist, the way Bazel does force linking is to find and pass all the object files directly to linker, the relevant code is here. You can also tell Bazel not to use /WHOLEARCHIVE option by export NO_WHOLE_ARCHIVE_OPTION=1

pcloudy@PCLOUDY1-W MSYS ~/workspace/buzel

$ bazel shutdown # This will cause Bazel to re-configure cc toolchain the next you run bazel build

pcloudy@PCLOUDY1-W MSYS ~/workspace/buzel

$ export NO_WHOLE_ARCHIVE_OPTION=1

pcloudy@PCLOUDY1-W MSYS ~/workspace/buzel

$ bazel build :link_crasher

.........................

INFO: Analysed target //:link_crasher.

INFO: Found 1 target...

INFO: From Linking link_crasher.exe:

   Creating library bazel-out/vc_14_0_x64-opt/bin/link_crasher.lib and object bazel-out/vc_14_0_x64-opt/bin/link_crasher.exp

Target //:link_crasher up-to-date:

  C:/temp/_bazel_pcloudy/k-8MzLG1/execroot/buzel/bazel-out/vc_14_0_x64-opt/bin/link_crasher.exe

INFO: Elapsed time: 39.739s, Critical Path: 2.44s

INFO: Build completed successfully, 4 total actions

pcloudy@PCLOUDY1-W MSYS ~/workspace/buzel

$ cat ./bazel-out/vc_14_0_x64-opt/bin/link_crasher.exe-2.params.msvc

/nologo

/MACHINE:X64

/SUBSYSTEM:CONSOLE

/OUT:bazel-out/vc_14_0_x64-opt/bin/link_crasher.exe

/DLL

ws2_32.lib

/MT

C:\temp\_bazel_pcloudy\k-8MzLG1\execroot\buzel\bazel-out\vc_14_0_x64-opt\bin\_objs\link_crasher\one.o

As you can see, when not using /WHOLEARCHIVE option, the linking action succeed.

And if you want build a shared library (a dll) on Windows, I would recommend not to add `-shared` into linkopts, instead, define the target like this:

cc_binary(

  name = "libok.dll",

  linkshared = 1,

  srcs = [

    "one.cc",

  ],

  linkopts = [

    "-Wl,ws2_32.lib",

  ],

)

The reason is we intend to remove the translation from gcc option to MSVC option in the long term.

Thanks,

Yun

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/5a0c22f8-35fd-42be-83c4-b8e9f46ba996%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[yury.p...@gmail.com]

Hi Yun,

Thank you very much for such comprehensive reply. Unfortunately NO_WHOLE_ARCHIVE_OPTION did not do the trick for me. I updated to bazel from 0.4.0 to 0.4.1 with no luck to build all buzel targets. Probably this relates to the particular combination of OS/linker I have: "Win10 1607 build 14393.447" and "Visual Studio Community 2015 Version 14.0.25431.01 Update 3". I might be wrong but I have impression that this problem appeared one day after windows update did a job. Anyway I can live with this MS linker bug simply working it around using #pragma.

By the way. Does bazel support CUDA right out of the box now? If so, is it available for all platforms, e.g. windows, mac and linux?

Many thanks for your help. I am really excited seeing huge progress in bazel windows support.

Best regards

Yury

[Yun Peng]

Oh, sorry, there seems to be something wrong with our 0.4.1 release, my change for adding NO_WHOLE_ARCHIVE_OPTION didn't make into the release.

But, you can try with 0.4.2-rc2.

Cuda support is complicated story, the short anwser is no, Bazel doesn't support CUDA for now. :(

Basically Tensorflow has a customized CROSSTOOL for CUDA build for Linux and Mac and also a cuda_configura.bzl for creating an external repository for cuda libraries. In the CROSSTOOL, they use a python wrapper script to parse gcc options and construct command line for nvcc compiler. And I added the same logic into Bazel's MSVC python wrapper script, but the cuda configuration still stays in TF repo, so it only works with TensorFlow. Hopefully one day we can move it into Bazel, so Bazel can support cuda build on Windows for any project. There are a lot of discussion about how should we improve our CROSSTOOL to better support this(using wrapper script to translate all the options is apparently not a nice way), but we don't have a very clear plan right now.

But stay tuned! :)

Cheers,

Yun

--

You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/4db580d1-7589-48cb-ae1b-50836d486a0f%40googlegroups.com.