[llvm-dev] ThinLTO Problem

[chenmindong via llvm-dev]

Hi,

I'm working on enabling thinLTO for our custom backend on LLVM-8 with lld to get code size benefits from dead symbol elimination. The code in LTO::run() of LTO.cpp confuses me that, even though thinLTO is specified, runRegularLTO() will be run first and its return value determines whether runThinLTO() will be executed.

My question is if it's clearly known that thinLTO is used, is it still necessary to execute runRegularLTO()?If it is, what's the reason behind? For now our custom backend, where distributed thinLTO is adopted, it works fine as I removed the line executing runRegularLTO(). But if I preserve it, the code fails the `if (Conf.PostInternalizeModuleHook &&!Conf.PostInternalizeModuleHook(0, *RegularLTO.CombinedModule))`, which I also don't understand, and fall through to backend() and abort there. I believe something is missed during adding the target support but cannot figure it out. Could anyone help?

Regards,
Mindong Chen
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Teresa Johnson via llvm-dev]

Hi Mindong,

On Wed, Nov 27, 2019 at 3:29 AM chenmindong via llvm-dev <llvm...@lists.llvm.org> wrote:

Hi,

I'm working on enabling thinLTO for our custom backend on LLVM-8 with lld to get code size benefits from dead symbol elimination. The code in LTO::run() of LTO.cpp confuses me that, even though thinLTO is specified, runRegularLTO() will be run first and its return value determines whether runThinLTO() will be executed.

My question is if it's clearly known that thinLTO is used, is it still necessary to execute runRegularLTO()?If it is, what's the reason behind?

This is to handle the case where the LTO link is given a mix of regular and thin LTO bitcode - it should do regular LTO on that subset and ThinLTO on the other. The other case this handles is when a bitcode object is split into regular and thin LTO halves - this is enabled for things like CFI but shouldn't be the default currently (you'd have to build with -fsplit-lto-unit to get it unless you are building with CFI).

Whether the bitcode is added to the regular or Thin LTO partition is determined in LTO::addModule, and is based on a flag set when the bitcode is read which is based in turn on whether the bitcode has a summary block, and whether that is a thinlto or regular (full) LTO summary block. How are you creating your bitcode files? If you run llvm-dis on it does it have summary entries? You can also see if you run llvm-bcanalyzer whether it has a GLOBALVAL_SUMMARY_BLOCK or a FULL_LTO_GLOBALVAL_SUMMARY_BLOCK or neither. If you are compiling with -flto=thin you should realistically have the former, which would make it a ThinLTO bitcode. But it sounds like you have some objects that either don't or have the full LTO summary.

For now our custom backend, where distributed thinLTO is adopted, it works fine as I removed the line executing runRegularLTO(). But if I preserve it, the code fails the `if (Conf.PostInternalizeModuleHook &&!Conf.PostInternalizeModuleHook(0, *RegularLTO.CombinedModule))`, which I also don't understand, and fall through to backend() and abort there. I believe something is missed during adding the target support but cannot figure it out. Could anyone help?

By default there should not be a PostInternalizeModuleHook set (it is set to support cases like -save-temps), so it isn't surprising that it would fail that test and fall through to the backend() call, which is what you would want if there was a regular LTO partition. Where is it aborting in the backend?

Teresa

Regards,
Mindong Chen
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

--

Teresa Johnson |

Software Engineer |

tejo...@google.com |

[David Blaikie via llvm-dev]

On Wed, Nov 27, 2019 at 6:29 AM chenmindong via llvm-dev <llvm...@lists.llvm.org> wrote:

Hi,

I'm working on enabling thinLTO for our custom backend on LLVM-8 with lld to get code size benefits from dead symbol elimination.

If you only want this - is -ffunction-sections -fdata-sections -Wl,-gc-sections sufficient?
 

[chenmindong via llvm-dev]

Hi David,

 

`-ffunction-sections -fdata-sections -Wl,-gc-sections ` is actually our first choice but because `-ffunction-sections` puts each function into its own section and section alignment cost is not negligible for our target, the gain is unpredictable and we deprecated the plan.

 

Regards,

Mindong

[chenmindong via llvm-dev]

Hi Teresa,

 

Thanks for the detailed reply!

> How are you creating your bitcode files?

I create the bitcode with `-flto=thin -c` and sure it has a GLOBALVAL_SUMMARY_BLOCK. And there’s no RegularLTO partition only ThinLTO bicode.

> Where is it aborting in the backend?

It aborts at ` report_fatal_error("Failed to setup codegen")` in of codegen() of LTOBackend.cpp. And before that in createTargetMachine() it also warns ‘xxx is not a recognized processor’. But the program should return before run into backend() IIUC since all I want is using lld with `-thinlto-index-only` option to generate *.thinlto.bc and feed it and bitocde to clang to invoke target-specific optimizations(we use gnu as and ld).Do you have any suggestion about this?

 

Regards,

Mindong

 

From: Teresa Johnson [mailto:tejo...@google.com]
Sent: Wednesday, November 27, 2019 11:12 PM
To: chenmindong <chenmi...@huawei.com>
Cc: llvm...@lists.llvm.org; Yuchao (Michael) <michael...@huawei.com>
Subject: Re: [llvm-dev] ThinLTO Problem

 

Hi Mindong,

[Teresa Johnson via llvm-dev]

On Wed, Nov 27, 2019 at 10:36 PM chenmindong <chenmi...@huawei.com> wrote:

Hi Teresa,

 

Thanks for the detailed reply!

> How are you creating your bitcode files?

I create the bitcode with `-flto=thin -c` and sure it has a GLOBALVAL_SUMMARY_BLOCK. And there’s no RegularLTO partition only ThinLTO bicode.

> Where is it aborting in the backend?

It aborts at ` report_fatal_error("Failed to setup codegen")` in of codegen() of LTOBackend.cpp. And before that in createTargetMachine() it also warns ‘xxx is not a recognized processor’. But the program should return before run into backend() IIUC since all I want is using lld with `-thinlto-index-only` option to generate *.thinlto.bc and feed it and bitocde to clang to invoke target-specific optimizations(we use gnu as and ld).Do you have any suggestion about this?

Ah, ok, that's helpful. So the report_fatal_error is almost certainly just a consequence of the earlier failure in createTargetMachine. And the issue is that it will try to code through the codegen path for regular LTO even if CombinedModule is empty, which it presumably is in your case. Even when LTO linking with -thinlto-index-only need to go through this path in case there are some regularLTO modules. And you must not be setting up any valid target machine, so it is complaining. Here's a few possible fixes/workarounds:

1) Can you pass a valid target machine that LLVM understands, even if you are using another tool downstream to compile for a specialized target?

2) If you are using LTO via the gold plugin (e.g. either gold or gnu), then you can pass the disable-output plugin option (i.e. -Wl,-plugin-opt,disable-output) to get it the regular LTO path to exit before codegen, without emitting anything for the (presumably empty) regular LTO object. Unfortunately, it does not look like lld supports this feature, although it would be straightforward to add. What linker are you using for the LTO link?

3) Supported by both the gold plugin (used by gold and gnu LTO), as well as lld, you can exit just slightly later using the following option, which will emit the regular LTO bitcode to the specified output file (which again would presumably be an "empty" module in your case) and exit, just before codegen: -Wl,-plugin-opt,emit-llvm

Teresa

[chenmindong via llvm-dev]

Hi Teresa,

 

The third way you suggested has solved my problem perfectly! `-plugin-opt emit-llvm` assigns PostInternalizeModuleHook a lambda which always returns false so that `if (Conf.PostInternalizeModuleHook && !Conf.PostInternalizeModuleHook(0, *RegularLTO.CombinedModule))` is met now and runRegularLTO() returns without executing backend(), though it creates an output file(defaults to be a.out), it doesn’t really matter. For the first suggestion, I need some time to figure out what a “valid target machine” requires here and what I missed. Your help is really appreciated!