[llvm-dev] Emit LLVM bitcode after ThinLTO

[Stephan Z via llvm-dev]

Hi,

I wanted to get the linked result in LLVM bitcode format.

With LTO, this can be done by -flto -Wl,-plugin-opt,emit-llvm. Instead of generating native executables, it outputs a file with bitcode format.

Does this still work with -flto=thin? -flto=thin -Wl,-plugin-opt,emit-llvm outputs a bitcode file, but its file size is too small, and does not contain all contents.

What is the correct way to emit LLVM bitcode after ThinLTO?

Thank you, S

[David Blaikie via llvm-dev]

I think there is a way, but I'm not entirely sure.

It's possible you've got the right thing & it's just that ThinLTO's
optimized away most of the file you're looking at?

> _______________________________________________
> LLVM Developers mailing list
> llvm...@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Teresa Johnson via llvm-dev]

Looks like I forgot to reply all so my reply and our follow on conversation got lost. Copied it below. Basically, this is expected because -emit-llvm only emits the result of the regular LTO link, which is typically empty for a normal ThinLTO build.

Teresa

On Sun, Jul 12, 2020 at 10:16 AM Teresa Johnson <tejo...@google.com> wrote:

On Sun, Jul 12, 2020 at 10:05 AM Stephan Z <stephan.y...@gmail.com> wrote:

Hi Teresa,

Thank you for your quick response. 

I have a program that analyzes the bitcode generated from "-flto -Wl,-plugin-opt,emit-llvm" to analyze code across modules. 

"-flto=thin -Wl,-plugin-opt,save-temps" dumps those bc files for each module. This does not provide a bitcode for all modules.

I was trying ThinLTO because LTO uses too much memory and is slow, and ThinLTO blog shows it uses less memory and time.

With ThinTLO, do we still have a way to get the entire bitcode after link? This allows reusing my program. 

Or does ThinLTO recommend other ways to do cross module analysis?

ThinLTO is fast and scalable because it does not combine all bitcode into a single module. It does whole program analysis on compact summaries of the input modules (during the "thin link"). You'd have to rewrite your analysis to operate on the combined summary index, either during the thin link, or on the serialized combined index (*.index.bc dumped by the save-temps option). But depending on what you are doing, you might need to enhance the summary info. See ModuleSummaryIndex.h for the info it currently has.

Teresa

Thank you, S

On Sun, Jul 12, 2020 at 8:13 AM Teresa Johnson <tejo...@google.com> wrote:

-emit-llvm only emits the result of the regular LTO link, which is why it is very small for your ThinLTO link (which supports having some amount of regular LTO objects mixed in, and also split thin and regular LTO bitcode modules, but by default for ThinLTO this regular LTO combined module will be small and effectively empty).

For ThinLTO there are going to be multiple modules. To get the bitcode for them you could use  -Wl,-plugin-opt,save-temps. This will give you more than you want, as it dumps the bitcode for each module at multiple places in the ThinLTO backends. The files end in .bc and the suffixes include the name of the phase where they were emitted:

- preopt.bc  (before any optimization)

- promote.bc (after ThinLTO local to global promotion)

- internalize.bc (after internalization)

- import.bc (after ThinLTO importing)

- opt.bc (after optimization pipeline but before codegen)

- precodegen.bc (usually the same as opt.bc)

Are you debugging or trying to do something else?

Teresa

On Mon, Jul 20, 2020 at 12:42 PM David Blaikie <dbla...@gmail.com> wrote:

I think there is a way, but I'm not entirely sure.

It's possible you've got the right thing & it's just that ThinLTO's
optimized away most of the file you're looking at?

On Sat, Jul 11, 2020 at 9:53 PM Stephan Z via llvm-dev
<llvm...@lists.llvm.org> wrote:
>
> Hi,
>
> I wanted to get the linked result in LLVM bitcode format.
>
> With LTO, this can be done by -flto -Wl,-plugin-opt,emit-llvm. Instead of generating native executables, it outputs a file with bitcode format.
>
> Does this still work with -flto=thin? -flto=thin -Wl,-plugin-opt,emit-llvm outputs a bitcode file, but its file size is too small, and does not contain all contents.
>
> What is the correct way to emit LLVM bitcode after ThinLTO?
>
> Thank you, S
>
> _______________________________________________
> LLVM Developers mailing list
> llvm...@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

--

Teresa Johnson |

Software Engineer |

tejo...@google.com |