[llvm-dev] Shipping LLVM.dll for the C API with the Windows installer.

[Jakob Bornecrantz via llvm-dev]

Hello list!

So I'm wondering if there is a will to ship a DLL with the C API
exported? LLVM currently ships with LTO.dll which has some C api
functions exported, made from the export file tools/lto/lto.exports,
so it would not be the first DLL LLVM shipped.

Currently I (and the users of my project[1]) are building it ourselves
using this script[2] derived from the LLVMSharp script[3]. Which adds
a extra long step for all users of ours and other projects like
LLVMSharp in order to use them.

The resulting LLVM.dll exports 809 functions and weighs in at around
18mb, so it would make the installer larger. There is also the cost of
maintaining a lists of export for the DLL, but hopefully we can use
the llvm-echo to test it on the windows nodes so any failure to add
new exports gets detected early.

Now for the bike-shed questions, do we call it LLVM-c.dll or just
LLVM.dll? Annotate all functions with a LLVM_EXPORT define or use a
llvm-c.exports file?

Thoughts and feedback welcome.

Cheers, Jakob.

[1] http://volt-lang.org
[2] https://github.com/VoltLang/GenLLVMDLLPy/blob/master/GenLLVMDLL.py
[3] https://github.com/Microsoft/LLVMSharp/blob/master/tools/GenLLVMDLL.ps1
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Jakob Bornecrantz via llvm-dev]

On Thu, Mar 30, 2017 at 4:46 PM, Jakob Bornecrantz <wallb...@gmail.com> wrote:
> Hello list!
>
> So I'm wondering if there is a will to ship a DLL with the C API
> exported? LLVM currently ships with LTO.dll which has some C api
> functions exported, made from the export file tools/lto/lto.exports,
> so it would not be the first DLL LLVM shipped.
>
> Currently I (and the users of my project[1]) are building it ourselves
> using this script[2] derived from the LLVMSharp script[3]. Which adds
> a extra long step for all users of ours and other projects like
> LLVMSharp in order to use them.
>
> The resulting LLVM.dll exports 809 functions and weighs in at around
> 18mb, so it would make the installer larger. There is also the cost of
> maintaining a lists of export for the DLL, but hopefully we can use
> the llvm-echo to test it on the windows nodes so any failure to add
> new exports gets detected early.
>
> Now for the bike-shed questions, do we call it LLVM-c.dll or just
> LLVM.dll? Annotate all functions with a LLVM_EXPORT define or use a
> llvm-c.exports file?
>
> Thoughts and feedback welcome.
>
> Cheers, Jakob.
>
> [1] http://volt-lang.org
> [2] https://github.com/VoltLang/GenLLVMDLLPy/blob/master/GenLLVMDLL.py
> [3] https://github.com/Microsoft/LLVMSharp/blob/master/tools/GenLLVMDLL.ps1

Ping? Any thoughts?

What would be the best way to proceed? My cmake-fu is very weak so any
pointers here from those knowledgeable and stakeholders in the code
would be greatly appreciated.

Cheers, Jakob.

[John Brawn via llvm-dev]

We already half-have this, the LLVM_BUILD_LLVM_C_DYLIB cmake option builds
a shared object which exports the llvm-c interface but it only works on
Darwin.

tools/llvm-shlib/CMakeLists.txt is where how this is done is defined, and it
looks like it does it by:
* build LLVM.so
* use nm+awk+sed to pick out the symbols starting with LLVM
* build LLVM-C.so using a -reexport_library linker option

I'm thinking that maybe we could make this not-darwin-specific by making
use of utils/extract_symbols.py:
* edit extract_symbols.py to add a --only_unmangled option (or maybe
--only_prefix=LLVM or something like that)
* on non-darwin use that instead of the existing command to get the
symbols to export (or: make extract_symbols.py work on Darwin)
* instead of building LLVM-C.so from LLVM.so using this -reexport_library
option, instead build LLVM-C.so from the same libraries that LLVM.so
is built from
(where I use .so above, it also applies to .dll)

John

[Jakob Bornecrantz via llvm-dev]

On Wed, Apr 5, 2017 at 5:20 PM, John Brawn <John....@arm.com> wrote:
> We already half-have this, the LLVM_BUILD_LLVM_C_DYLIB cmake option builds
> a shared object which exports the llvm-c interface but it only works on
> Darwin.
>
> tools/llvm-shlib/CMakeLists.txt is where how this is done is defined, and it
> looks like it does it by:
> * build LLVM.so
> * use nm+awk+sed to pick out the symbols starting with LLVM
> * build LLVM-C.so using a -reexport_library linker option
>
> I'm thinking that maybe we could make this not-darwin-specific by making
> use of utils/extract_symbols.py:
> * edit extract_symbols.py to add a --only_unmangled option (or maybe
> --only_prefix=LLVM or something like that)
> * on non-darwin use that instead of the existing command to get the
> symbols to export (or: make extract_symbols.py work on Darwin)
> * instead of building LLVM-C.so from LLVM.so using this -reexport_library
> option, instead build LLVM-C.so from the same libraries that LLVM.so
> is built from
> (where I use .so above, it also applies to .dll)
>
> John

Thanks for the info John!

I tried to enable LLVM_BUILD_LLVM_C_DYLIB on my linux machine
to play around a little bit. I disabled the APPLE check to get past that.
The config failed if I had BUILD_SHARED_LIBS also set to true.
If I tried that without the flag it complained about missing:
builddir/./lib/libLLVM.so

Which is a bit weird since it builds a lib/libLLVM-5.0svn.so.

I have to admit I'm a tiny bit in over my head here with the cmake scripts here.

Cheers, Jakob.

[Alexander Benikowski via llvm-dev]

The following is an older commandline i used. Have a more recent one at home. But basically you can write it as batch and trigger it within a target during the build(never got targets into correct order, i am a cmake noob)

So for reference, i'll post this one and look for the recent one at home(if that didn't go down with my recent hdd crash):

cmd /Q /V:ON /c "for /F "tokens=4" %l in ('dir llvm*.lib') do (for /F "tokens=2" %e in ('dumpbin /linkermember:1 %l ^| findstr "_LLVM"') do (set symbolname=%e & echo !symbolname:~1!))"

What it does:

For all *.lib files ina specific dir starting with LLVM call dumpbin.exe (tool from Visualstudio) and write all symbols with LLVM_ into a specific file

[Alexander Benikowski via llvm-dev]

Maybe someone can use this as a startingpoint to add the windows-specific commandblock which is triggered instead of the Darwin one together with a proper setup of targets to have the libs build before.

[Jakob Bornecrantz via llvm-dev]

Made a bit of headway here: https://reviews.llvm.org/D35077

Cheers, Jakob.

[Alexander Benikowski via llvm-dev]

Thanks for starting!

Btw here is my CMD version:

cmd /Q /V:ON /c "for /F "tokens=4" %l in ('dir llvm*.lib') do (for /F "tokens=2" %e in ('dumpbin /linkermember:1 %l ^| findstr "_LLVM"') do (set symbolname=%e & echo !symbolname:~1!))"

You run it in the directory with all the llvm*.lib (yes the ThinLTO is excluded in this example but can be added easily i think)

[Alexander Benikowski via llvm-dev]

I am sorry, just noticed i repeated myself :(

[Alexander Benikowski via llvm-dev]

The Review https://reviews.llvm.org/D35077

has been accepted ~3weeks ago. Is someone up for committing it? Just in case it felt under the radar.