-polly-position=before-vectorizer using lld-link.exe with thinlto

[robri...@gmail.com]

clang-cl.exe with thinlto enabled can enable the polly early and after-loopopt positions since those are in the (legacy) module pass manager via EP_ModuleOptimizerEarly and EP_LoopOptimizerEnd before the conditioned thinlto codegen passes.

AFAICT (?) clang-cl.exe does not enable -polly-position=before-vectorizer with thinlto enabled, as thinlto passes are deferred until lld-link.exe linking. addExtensionsToPM(EP_VectorizerStart, MPM); is conditioned on PerformThinLTO in the module pass manager as called by the thinlto pass manager.

lld-link.exe can pass various -mllvm options, but it does not recognize Polly arguments as being valid. I *assume* initializePollyPasses never actually happens for lld-link.exe directed thinlto passes since lld is not built against Polly like clang, opt, etc.

lld-link.exe -mllvm:-polly

lld-link (LLVM option parsing): Unknown command line argument '-polly'

[Michael Kruse]

There is currently a review on generalizing linking Polly into tools
(opt, bugpoint and clang: https://reviews.llvm.org/D61446). I haven't
used thinlto with Polly yet. Should "tools" therefore include
lld-link?

Michael

> --
> You received this message because you are subscribed to the Google Groups "Polly Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to polly-dev+...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/polly-dev/55bbdb08-5261-446e-a0ab-1697df73fdcb%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Tardyzentrismus verboten!

[robri...@gmail.com]

I would assume so, along with making sure lld/COFF/Driver.cpp is parsing -mllvm:-polly and probably that lld/COFF/LTO.cpp is passing the polly argument(s) along.

I also *assume* (there is that word again) that getting -polly-position=before-vectorizer into the thinlto pipeline should benefit from potentially improved interprocedural optimizations. :)

For comparison, polly in the lto pipeline is something not even polly+fulllto is doing right now. I do know Tobias G. previously was experimenting with polly in the fullto pipeline to enable polly interprocedural optimizations in libquantum.

https://reviews.llvm.org/D38585

https://reviews.llvm.org/D38584

BTW, it looks like the other LLD clients can parse -mllvm as well, so they probably also should include extension support if implementing D61446.

On Thursday, May 9, 2019 at 11:19:54 PM UTC-4, Michael Kruse wrote:

There is currently a review on generalizing linking Polly into tools
(opt, bugpoint and clang: https://reviews.llvm.org/D61446). I haven't
used thinlto with Polly yet. Should "tools" therefore include
lld-link?

Michael

Am Do., 9. Mai 2019 um 22:10 Uhr schrieb <robri...@gmail.com>:
>
> clang-cl.exe with thinlto enabled can enable the polly early and after-loopopt positions since those are in the (legacy) module pass manager via EP_ModuleOptimizerEarly and EP_LoopOptimizerEnd before the conditioned thinlto codegen passes.
>
> AFAICT (?) clang-cl.exe does not enable -polly-position=before-vectorizer with thinlto enabled, as thinlto passes are deferred until lld-link.exe linking. addExtensionsToPM(EP_VectorizerStart, MPM); is conditioned on PerformThinLTO in the module pass manager as called by the thinlto pass manager.
>
> lld-link.exe can pass various -mllvm options, but it does not recognize Polly arguments as being valid. I *assume* initializePollyPasses never actually happens for lld-link.exe directed thinlto passes since lld is not built against Polly like clang, opt, etc.
>
> lld-link.exe -mllvm:-polly
> lld-link (LLVM option parsing): Unknown command line argument '-polly'
>
> --
> You received this message because you are subscribed to the Google Groups "Polly Development" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to poll...@googlegroups.com.

[Michael Kruse]

Could you add a comment about this in https://reviews.llvm.org/D61446 ?

Michael

> To unsubscribe from this group and stop receiving emails from it, send an email to polly-dev+...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/polly-dev/915f5cb2-381c-420c-a8f6-ef8b5b56db26%40googlegroups.com.

[robri...@gmail.com]

Hi Michael,

I do not have a phab account at this time. If D61446 lands, it would seem (hopefully?) trivial to add polly as a lld extension.

Though lld probably is going to need some work, too. Once the extension support lands, I suspect actually registering and initializing polly passes is going to require lld pulling in llvm passes (npm) and ipo (pm) like opt and similar does.

[Michael Kruse]

I added a comment to D61446.At the moment, the patch does not change lld yet.

Michael

> To unsubscribe from this group and stop receiving emails from it, send an email to polly-dev+...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/polly-dev/0ebb6edb-9870-446a-b035-3ef8f09e02e5%40googlegroups.com.

[robri...@gmail.com]

Finally got around to looking at running Polly during LTO via LLD.

A few quick-and-dirty edits got Polly working during ThinLTO codegen via lld-link.exe on Windows.

D:\src_builds\llvm-monorepo\llvm\lld\COFF\CMakeLists.txt

Add IPO to LINK_COMPONENTS. Add Polly to LINK_LIBS. File attached.

D:\src_builds\llvm-monorepo\llvm\lld\COFF\Driver.cpp

Add namespace for Polly. Line 57 in attached file.

Initialize Polly for LTO. Line 1113 in attached file.

lld-link.exe will now accept, initialize, and pass arguments to Polly.

-mllvm:-polly -mllvm:-pass-remarks-analysis=polly

Example of analysis remarks generated by lld-link.exe for some code I compiled with both Polly and ThinLTO enabled.

....

<unknown>:0:0: SCoP begins here.

<unknown>:0:0: Inbounds assumption:     [p_0, p_1, p_2, p_3] -> {  : p_3 >= p_0 }

<unknown>:0:0: Inbounds assumption:     [p_0, p_1, p_2, p_3] -> {  : p_2 >= 2p_0 }

<unknown>:0:0: Inbounds assumption:     [p_0, p_1, p_2, p_3] -> {  : p_2 >= 2p_0 }

<unknown>:0:0: Inbounds assumption:     [p_0, p_1, p_2, p_3] -> {  : p_2 >= -1 + 2p_0 }

<unknown>:0:0: Inbounds assumption:     [p_0, p_1, p_2, p_3] -> {  : p_2 >= -1 + 2p_0 }

<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 >= 4611686018427387905 and p_1 > 0 }

<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 >= 4611686018427387905 and p_1 > 0 }

<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 >= 4611686018427387905 and p_1 > 0 }

<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 > 0 and p_1 >= 4611686018427387905 }

<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 > 0 and p_1 >= 4611686018427387905 }

<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 > 0 and p_1 >= 4611686018427387905 }

<unknown>:0:0: Finite loop restriction: [p_0, p_1, p_2, p_3] -> {  : p_0 < 0 and p_1 > 0 }

<unknown>:0:0: Finite loop restriction: [p_0, p_1, p_2, p_3] -> {  : p_1 < 0 and (p_0 < 0 or p_0 > 0) }

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: SCoP ends here.

....

I would suspect similar can be done for ELF and whatever else. Assuming someone wants to add some build conditionals and submit the needed patch(es).

[Michael Kruse]

Could you upload a patch to Phabricator? Applying the changes manually
is quite cumbersome.

Michael

> To unsubscribe from this group and stop receiving emails from it, send an email to polly-dev+...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/polly-dev/8b14ede0-3716-4b8d-8ae1-d2085bdbba26%40googlegroups.com.



--
Tardyzentrismus verboten!

[robri...@gmail.com]

Hi Michael,

Thanks for the reply. I am not a project dev, but it is just a couple of very minor edits since r24ab9b537e61 has landed. :) No extra conditionals or similar stuff now needed.

If Serge or you want to land a quick patch:

1. Add ENABLE_PLUGINS to lld\COFF\CMakeLists.txt

2. Add

#define HANDLE_EXTENSION(Ext)                                                  \

  llvm::PassPluginLibraryInfo get##Ext##PluginInfo();

#include "llvm/Support/Extension.def"

to lld\COFF\Driver.cpp

3. Build LLVM with Polly and Polly linked into tools. Ex:

cmake -DLLVM_TARGETS_TO_BUILD=X86 -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang;lld;polly;compiler-rt -DLLVM_POLLY_LINK_INTO_TOOLS=ON  -G "Ninja" ..\llvm\llvm

4. Be sure to pass Polly into lld-link.exe using -mllvm:-polly argument.

I do not have a Linux box in front of me at the moment, but I suspect the ELF side of LLD should work with the same couple of code edits.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/polly-dev/8b14ede0-3716-4b8d-8ae1-d2085bdbba26%40googlegroups.com.



--
Tardyzentrismus verboten!

[robri...@gmail.com]

Just noticed I forgot to copy-and-paste a needed include for lld\COFF\Driver.cpp:

#include "llvm/Passes/PassPlugin.h"

[robri...@gmail.com]

Just noticed it is working here now without manually patching lld-link.exe:

"-mllvm:-polly",

"-mllvm:-pass-remarks-analysis=polly",

[....]

LINK brotli.exe

<unknown>:0:0: SCoP begins here.

<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2] -> {  : p_0 >= 2305843009213693953 }

<unknown>:0:0: Signed-unsigned restriction:     [p_0, p_1, p_2] -> {  : p_2 < 0 }

<unknown>:0:0: Signed-unsigned restriction:     [p_0, p_1, p_2] -> {  : p_1 < 0 }

<unknown>:0:0: Signed-unsigned restriction:     [p_0, p_1, p_2] -> {  : p_2 < 0 }

<unknown>:0:0: Signed-unsigned restriction:     [p_0, p_1, p_2] -> {  : p_2 < 0 }

<unknown>:0:0: Signed-unsigned restriction:     [p_0, p_1, p_2] -> {  : p_1 < 0 }

<unknown>:0:0: Finite loop restriction: [p_0, p_1, p_2] -> {  : p_0 < 0 }

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.

<unknown>:0:0: SCoP ends here.

[....]

Thanks!