[llvm-dev] how to add my own passes to LTO pass

[y liu via llvm-dev]

Hi,

I have a module pass and I hope to use it to optimize a real-world program.

I need LTO，and I have got LTO plugin.

But How can I add my passes to LTO Pass. I can't find solution. 

What should I do?

[Teresa Johnson via llvm-dev]

On Wed, Apr 22, 2020 at 5:12 AM y liu via llvm-dev <llvm...@lists.llvm.org> wrote:

Hi,

I have a module pass and I hope to use it to optimize a real-world program.

I need LTO，and I have got LTO plugin.

Do you mean the LLVM gold plugin (relevant when doing LTO links with gold or gnu ld), or are you trying to load your new pass via a pass plugin?

 

But How can I add my passes to LTO Pass. I can't find solution. 

What should I do?

This depends on whether you are trying to add via a pass plugin dynamically via the command line (I believe the support for this is work in progress), or by manually modifying the pass pipeline within LLVM.

Have you implemented as an old pass manager pass (the current default) or the new pass manager or both?

Is it an LTO specific pass, or do you also want to run for non-LTO optimizing compiles (most optimizing passes should run in all cases).

Assuming the latter, and also assuming you are using the default old pass manager, you would want to look at the following places in PassManagerBuilder.cpp:

regular (full) LTO: PassManagerBuilder::addLTOOptimizationPasses

ThinLTO: PassManagerBuilder::populateModulePassManager, which is used both by the pre-LTO compile and by the LTO linking ThinLTO compiles. If you want to insert your pass for both, just put it in the relevant place in this file, if you want to only do during the LTO link portion of the ThinLTO compile, then guard it with a check for "PerformThinLTO".

For the new pass manager, it would be added to PassBuilder.cpp (that is structured a bit differently so let me know and I can point you in the right direction). If you are planning to send upstream it should be supported by both pass managers.

HTH,

Teresa

_______________________________________________

LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

--

Teresa Johnson |

Software Engineer |

tejo...@google.com |

[Teresa Johnson via llvm-dev]

cc-ed llvm-dev back so others can hopefully help. I don't load passes via plugins so this is outside my area of expertise. But I have seen some recent patches fly by that are working on this.

I saw from an earlier question you posted that you are using gold. There is a patch under development/review to get pass plugins to work for LTO through gold. See https://reviews.llvm.org/D77704.

Eli (added) was working on adding support as well, here are a couple of patches he recently landed for this in LTO and lld:

https://reviews.llvm.org/D75879 (lld)

https://reviews.llvm.org/D76866 (LTO)

I'm not completely sure what the status is at this point though. Eli is probably the best person to answer.

Teresa

On Wed, Apr 22, 2020 at 7:50 PM y liu <ly82...@gmail.com> wrote:

I‘m using LLVM 10, and register my pass by following code:

char MyModulePass::ID = 0;

static RegisterPass<MyModulePass> X("MyModule", "MyModule pass");

static void registerMyModulePass(const PassManagerBuilder &,

                         legacy::PassManagerBase &PM) {

  PM.add(new MyModulePass());

}

static RegisterStandardPasses

  RegisterMyPass(PassManagerBuilder::EP_FullLinkTimeOptimizationLast,

                 registerMyModulePass);

MyModulePass reles on helper.c.

I want to use it for optimization after the link is compelte.

Now, I compile it manually with opt.

For example:

Target: fun1.c fun2.c fun3.c main.c
Pass: Mypass.so
Dependency: helper.c

fun1.c fun2.c fun3.c main.c ---> target.bc
                   helper.c ---> helper.o

target.bc helper.o ---> target-link.bc ----> opt Mypass.so  ---> target-opt.bc ---> target

I want to achieve my goals by LTO.

Teresa Johnson <tejo...@google.com>于2020年4月22日 周三23:11写道：

On Wed, Apr 22, 2020 at 5:12 AM y liu via llvm-dev <llvm...@lists.llvm.org> wrote:

Hi,

I have a module pass and I hope to use it to optimize a real-world program.

I need LTO，and I have got LTO plugin.

Do you mean the LLVM gold plugin (relevant when doing LTO links with gold or gnu ld), or are you trying to load your new pass via a pass plugin?

 Yes, it’s gold plugin, I need it to link .bc and .o files. I want to optimize the target after linking. 

But How can I add my passes to LTO Pass. I can't find solution. 

What should I do?

This depends on whether you are trying to add via a pass plugin dynamically via the command line (I believe the support for this is work in progress), or by manually modifying the pass pipeline within LLVM.

Have you implemented as an old pass manager pass (the current default) or the new pass manager or both?

Is it an LTO specific pass, or do you also want to run for non-LTO optimizing compiles (most optimizing passes should run in all cases).

Assuming the latter, and also assuming you are using the default old pass manager, you would want to look at the following places in PassManagerBuilder.cpp:

regular (full) LTO: PassManagerBuilder::addLTOOptimizationPasses

ThinLTO: PassManagerBuilder::populateModulePassManager, which is used both by the pre-LTO compile and by the LTO linking ThinLTO compiles. If you want to insert your pass for both, just put it in the relevant place in this file, if you want to only do during the LTO link portion of the ThinLTO compile, then guard it with a check for "PerformThinLTO".

For the new pass manager, it would be added to PassBuilder.cpp (that is structured a bit differently so let me know and I can point you in the right direction). If you are planning to send upstream it should be supported by both pass managers.

HTH,

Teresa

_______________________________________________

LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

--

Teresa Johnson |

Software Engineer |

tejo...@google.com |

[Eli Friedman via llvm-dev]

It should be possible to dynamically load legacy pass manager plugins into lld on trunk.  Pass “-mllvm=-load=plugin.so” to lld (“-fuse-ld=lld -Wl,-mllvm=-load=plugin.so” to clang) for a plugin with an appropriate RegisterStandardPasses, and it should be called.

 

Dynamically loading pass plugins into gold is still in progress. (See https://reviews.llvm.org/D77704.)

 

It’s also possible to statically link both legacy pass manager and new pass manager LTO pass plugins into LLVM; that should work for both gold and lld without any command-line flags at runtime.

 

llvm/examples/Bye/ is a very simple pass plugin which supports running during LTO; it should serve as a reasonable template if you’re having trouble getting started.  It support both the legacy pass manager and the new pass manager; new pass manager isn’t enabled by default, so you can ignore the getByePluginInfo() etc. if you’re not interested in that.

 

There’s a little documentation for pass plugins at http://llvm.org/docs/WritingAnLLVMPass.html#building-pass-plugins ; the documentation could probably use some work.

 

Note that all of this is only working on trunk; 10.0 doesn’t have the changes in question.

 

-Eli

[y liu via llvm-dev]

  Thanks for your help.   

Eli Friedman <efri...@quicinc.com> 于2020年4月24日周五 上午2:59写道：