-polly-position=before-vectorizer using lld-link.exe with thinlto

85 views
Skip to first unread message

robri...@gmail.com

unread,
May 9, 2019, 11:10:40 PM5/9/19
to Polly Development
clang-cl.exe with thinlto enabled can enable the polly early and after-loopopt positions since those are in the (legacy) module pass manager via EP_ModuleOptimizerEarly and EP_LoopOptimizerEnd before the conditioned thinlto codegen passes.

AFAICT (?) clang-cl.exe does not enable -polly-position=before-vectorizer with thinlto enabled, as thinlto passes are deferred until lld-link.exe linking. addExtensionsToPM(EP_VectorizerStart, MPM); is conditioned on PerformThinLTO in the module pass manager as called by the thinlto pass manager.

lld-link.exe can pass various -mllvm options, but it does not recognize Polly arguments as being valid. I *assume* initializePollyPasses never actually happens for lld-link.exe directed thinlto passes since lld is not built against Polly like clang, opt, etc.

lld-link.exe -mllvm:-polly
lld-link (LLVM option parsing): Unknown command line argument '-polly'

Michael Kruse

unread,
May 9, 2019, 11:19:54 PM5/9/19
to robri...@gmail.com, Polly Development
There is currently a review on generalizing linking Polly into tools
(opt, bugpoint and clang: https://reviews.llvm.org/D61446). I haven't
used thinlto with Polly yet. Should "tools" therefore include
lld-link?

Michael
> --
> You received this message because you are subscribed to the Google Groups "Polly Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to polly-dev+...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/polly-dev/55bbdb08-5261-446e-a0ab-1697df73fdcb%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Tardyzentrismus verboten!

robri...@gmail.com

unread,
May 10, 2019, 3:33:15 AM5/10/19
to Polly Development
I would assume so, along with making sure lld/COFF/Driver.cpp is parsing -mllvm:-polly and probably that lld/COFF/LTO.cpp is passing the polly argument(s) along.

I also *assume* (there is that word again) that getting -polly-position=before-vectorizer into the thinlto pipeline should benefit from potentially improved interprocedural optimizations. :)

For comparison, polly in the lto pipeline is something not even polly+fulllto is doing right now. I do know Tobias G. previously was experimenting with polly in the fullto pipeline to enable polly interprocedural optimizations in libquantum.


BTW, it looks like the other LLD clients can parse -mllvm as well, so they probably also should include extension support if implementing D61446.

On Thursday, May 9, 2019 at 11:19:54 PM UTC-4, Michael Kruse wrote:
There is currently a review on generalizing linking Polly into tools
(opt, bugpoint and clang: https://reviews.llvm.org/D61446). I haven't
used thinlto with Polly yet. Should "tools" therefore include
lld-link?

Michael

Am Do., 9. Mai 2019 um 22:10 Uhr schrieb <robri...@gmail.com>:
>
> clang-cl.exe with thinlto enabled can enable the polly early and after-loopopt positions since those are in the (legacy) module pass manager via EP_ModuleOptimizerEarly and EP_LoopOptimizerEnd before the conditioned thinlto codegen passes.
>
> AFAICT (?) clang-cl.exe does not enable -polly-position=before-vectorizer with thinlto enabled, as thinlto passes are deferred until lld-link.exe linking. addExtensionsToPM(EP_VectorizerStart, MPM); is conditioned on PerformThinLTO in the module pass manager as called by the thinlto pass manager.
>
> lld-link.exe can pass various -mllvm options, but it does not recognize Polly arguments as being valid. I *assume* initializePollyPasses never actually happens for lld-link.exe directed thinlto passes since lld is not built against Polly like clang, opt, etc.
>
> lld-link.exe -mllvm:-polly
> lld-link (LLVM option parsing): Unknown command line argument '-polly'
>
> --
> You received this message because you are subscribed to the Google Groups "Polly Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to poll...@googlegroups.com.

Michael Kruse

unread,
May 22, 2019, 2:54:13 PM5/22/19
to robri...@gmail.com, Polly Development
Could you add a comment about this in https://reviews.llvm.org/D61446 ?

Michael
> To unsubscribe from this group and stop receiving emails from it, send an email to polly-dev+...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/polly-dev/915f5cb2-381c-420c-a8f6-ef8b5b56db26%40googlegroups.com.

robri...@gmail.com

unread,
May 28, 2019, 7:13:39 PM5/28/19
to Polly Development
Hi Michael,

I do not have a phab account at this time. If D61446 lands, it would seem (hopefully?) trivial to add polly as a lld extension.

Though lld probably is going to need some work, too. Once the extension support lands, I suspect actually registering and initializing polly passes is going to require lld pulling in llvm passes (npm) and ipo (pm) like opt and similar does.

Michael Kruse

unread,
May 29, 2019, 4:58:56 PM5/29/19
to robri...@gmail.com, Polly Development
I added a comment to D61446.At the moment, the patch does not change lld yet.

Michael
> To unsubscribe from this group and stop receiving emails from it, send an email to polly-dev+...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/polly-dev/0ebb6edb-9870-446a-b035-3ef8f09e02e5%40googlegroups.com.

robri...@gmail.com

unread,
Jan 1, 2020, 1:26:11 PM1/1/20
to Polly Development
Finally got around to looking at running Polly during LTO via LLD.

A few quick-and-dirty edits got Polly working during ThinLTO codegen via lld-link.exe on Windows.

D:\src_builds\llvm-monorepo\llvm\lld\COFF\CMakeLists.txt

Add IPO to LINK_COMPONENTS. Add Polly to LINK_LIBS. File attached.

D:\src_builds\llvm-monorepo\llvm\lld\COFF\Driver.cpp

Add namespace for Polly. Line 57 in attached file.

Initialize Polly for LTO. Line 1113 in attached file.

lld-link.exe will now accept, initialize, and pass arguments to Polly.

-mllvm:-polly -mllvm:-pass-remarks-analysis=polly

Example of analysis remarks generated by lld-link.exe for some code I compiled with both Polly and ThinLTO enabled.

....
<unknown>:0:0: SCoP begins here.
<unknown>:0:0: Inbounds assumption:     [p_0, p_1, p_2, p_3] -> {  : p_3 >= p_0 }
<unknown>:0:0: Inbounds assumption:     [p_0, p_1, p_2, p_3] -> {  : p_2 >= 2p_0 }
<unknown>:0:0: Inbounds assumption:     [p_0, p_1, p_2, p_3] -> {  : p_2 >= 2p_0 }
<unknown>:0:0: Inbounds assumption:     [p_0, p_1, p_2, p_3] -> {  : p_2 >= -1 + 2p_0 }
<unknown>:0:0: Inbounds assumption:     [p_0, p_1, p_2, p_3] -> {  : p_2 >= -1 + 2p_0 }
<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 >= 4611686018427387905 and p_1 > 0 }
<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 >= 4611686018427387905 and p_1 > 0 }
<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 >= 4611686018427387905 and p_1 > 0 }
<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 > 0 and p_1 >= 4611686018427387905 }
<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 > 0 and p_1 >= 4611686018427387905 }
<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2, p_3] -> {  : p_0 > 0 and p_1 >= 4611686018427387905 }
<unknown>:0:0: Finite loop restriction: [p_0, p_1, p_2, p_3] -> {  : p_0 < 0 and p_1 > 0 }
<unknown>:0:0: Finite loop restriction: [p_0, p_1, p_2, p_3] -> {  : p_1 < 0 and (p_0 < 0 or p_0 > 0) }
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: SCoP ends here.
....

I would suspect similar can be done for ELF and whatever else. Assuming someone wants to add some build conditionals and submit the needed patch(es).
CMakeLists.txt
Driver.cpp

Michael Kruse

unread,
Jan 3, 2020, 6:09:50 AM1/3/20
to robri...@gmail.com, Polly Development
Could you upload a patch to Phabricator? Applying the changes manually
is quite cumbersome.

Michael
> To unsubscribe from this group and stop receiving emails from it, send an email to polly-dev+...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/polly-dev/8b14ede0-3716-4b8d-8ae1-d2085bdbba26%40googlegroups.com.



--
Tardyzentrismus verboten!

robri...@gmail.com

unread,
Jan 3, 2020, 10:56:07 PM1/3/20
to Polly Development
Hi Michael,

Thanks for the reply. I am not a project dev, but it is just a couple of very minor edits since r24ab9b537e61 has landed. :) No extra conditionals or similar stuff now needed.

If Serge or you want to land a quick patch:

1. Add ENABLE_PLUGINS to lld\COFF\CMakeLists.txt

2. Add

#define HANDLE_EXTENSION(Ext)                                                  \
  llvm::PassPluginLibraryInfo get##Ext##PluginInfo();
#include "llvm/Support/Extension.def"

to lld\COFF\Driver.cpp

3. Build LLVM with Polly and Polly linked into tools. Ex:

cmake -DLLVM_TARGETS_TO_BUILD=X86 -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang;lld;polly;compiler-rt -DLLVM_POLLY_LINK_INTO_TOOLS=ON  -G "Ninja" ..\llvm\llvm

4. Be sure to pass Polly into lld-link.exe using -mllvm:-polly argument.

I do not have a Linux box in front of me at the moment, but I suspect the ELF side of LLD should work with the same couple of code edits.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/polly-dev/8b14ede0-3716-4b8d-8ae1-d2085bdbba26%40googlegroups.com.



--
Tardyzentrismus verboten!

robri...@gmail.com

unread,
Jan 13, 2020, 12:56:53 AM1/13/20
to Polly Development
Just noticed I forgot to copy-and-paste a needed include for lld\COFF\Driver.cpp:

#include "llvm/Passes/PassPlugin.h"

robri...@gmail.com

unread,
May 2, 2020, 7:20:33 AM5/2/20
to Polly Development
Just noticed it is working here now without manually patching lld-link.exe:

"-mllvm:-polly",
"-mllvm:-pass-remarks-analysis=polly",

[....]
LINK brotli.exe
<unknown>:0:0: SCoP begins here.
<unknown>:0:0: No-overflows restriction:        [p_0, p_1, p_2] -> {  : p_0 >= 2305843009213693953 }
<unknown>:0:0: Signed-unsigned restriction:     [p_0, p_1, p_2] -> {  : p_2 < 0 }
<unknown>:0:0: Signed-unsigned restriction:     [p_0, p_1, p_2] -> {  : p_1 < 0 }
<unknown>:0:0: Signed-unsigned restriction:     [p_0, p_1, p_2] -> {  : p_2 < 0 }
<unknown>:0:0: Signed-unsigned restriction:     [p_0, p_1, p_2] -> {  : p_2 < 0 }
<unknown>:0:0: Signed-unsigned restriction:     [p_0, p_1, p_2] -> {  : p_1 < 0 }
<unknown>:0:0: Finite loop restriction: [p_0, p_1, p_2] -> {  : p_0 < 0 }
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: Possibly aliasing pointer, use restrict keyword.
<unknown>:0:0: SCoP ends here.
[....]

Thanks!
Reply all
Reply to author
Forward
0 new messages