[llvm-dev] Is there a way to pass Optimization passes to clang?

[Phil Tomson via llvm-dev]

0 down vote favorite

I'm trying to debug an issue for a new target where a testcase fails with -O1 optimization and passes with -O0 optimization.

I got a list of optimization passes being performed when 'clang -O1' is called like this:

llvm-as < /dev/null | opt -O1 -disable-output -debug-pass=Arguments

Which results in:

Pass Arguments:  -no-aa -tbaa -targetlibinfo -basicaa -notti -preverify -domtree -verify -simplifycfg -domtree -sroa -early-cse -lower-expect
Pass Arguments:  -targetlibinfo -no-aa -tbaa -basicaa -notti -globalopt -ipsccp -deadargelim -instcombine -simplifycfg -basiccg -prune-eh -inline-cost -always-inline -functionattrs -sroa -domtree -early-cse -simplify-libcalls -lazy-value-info -jump-threading -correlated-propagation -simplifycfg -instcombine -tailcallelim -simplifycfg -reassociate -domtree -loops -loop-simplify -lcssa -loop-rotate -licm -lcssa -loop-unswitch -instcombine -scalar-evolution -loop-simplify -lcssa -indvars -loop-idiom -loop-deletion -loop-unroll -memdep -memcpyopt -sccp -instcombine -lazy-value-info -jump-threading -correlated-propagation -domtree -memdep -dse -adce -simplifycfg -instcombine -strip-dead-prototypes -preverify -domtree -verify

(BTW: why are there two Pass Arguments lists returned there?)

Now my intent is to figure out which optimization pass is causing the problem by trying each one until I hit the same problem as with -O1, but clang itself doesn't seem to allow these commandline options, for example:

clang -loops -lcssa ....

Results in:

clang: warning: -loops: 'linker' input unused
clang: warning: -lcssa: 'linker' input unused

(Makes sense as it conflicts with the -l command line opt) 
So is there a way to pass these in on the command line? (or bonus if 
there's a way to specify a file which contains a list of optimization 
passes to perform)

Note: I realize there must be a way to break everything up into stages and use opt at some point and pass the optimization command line options to it, but I've got a pretty large set of Makefiles which would need to change to do that.

[Mehdi Amini via llvm-dev]

Adding CC: cfe-dev 

On Dec 2, 2015, at 9:56 AM, Phil Tomson via llvm-dev <llvm...@lists.llvm.org> wrote:

0 down vote favorite

I'm trying to debug an issue for a new target where a testcase fails with -O1 optimization and passes with -O0 optimization.

I got a list of optimization passes being performed when 'clang -O1' is called like this:

llvm-as < /dev/null | opt -O1 -disable-output -debug-pass=Arguments

Which results in:

Pass Arguments:  -no-aa -tbaa -targetlibinfo -basicaa -notti -preverify -domtree -verify -simplifycfg -domtree -sroa -early-cse -lower-expect
Pass Arguments:  -targetlibinfo -no-aa -tbaa -basicaa -notti -globalopt -ipsccp -deadargelim -instcombine -simplifycfg -basiccg -prune-eh -inline-cost -always-inline -functionattrs -sroa -domtree -early-cse -simplify-libcalls -lazy-value-info -jump-threading -correlated-propagation -simplifycfg -instcombine -tailcallelim -simplifycfg -reassociate -domtree -loops -loop-simplify -lcssa -loop-rotate -licm -lcssa -loop-unswitch -instcombine -scalar-evolution -loop-simplify -lcssa -indvars -loop-idiom -loop-deletion -loop-unroll -memdep -memcpyopt -sccp -instcombine -lazy-value-info -jump-threading -correlated-propagation -domtree -memdep -dse -adce -simplifycfg -instcombine -strip-dead-prototypes -preverify -domtree -verify

(BTW: why are there two Pass Arguments lists returned there?)

This is because the middle-end and the backend are using a different PassManager (I don’t think there is any reason for doing this but that’s how it’s implemented in clang).

Now my intent is to figure out which optimization pass is causing the problem by trying each one until I hit the same problem as with -O1, but clang itself doesn't seem to allow these commandline options, for example:

clang -loops -lcssa ....

Results in:

clang: warning: -loops: 'linker' input unused
clang: warning: -lcssa: 'linker' input unused

(Makes sense as it conflicts with the -l command line opt) 
So is there a way to pass these in on the command line? (or bonus if 
there's a way to specify a file which contains a list of optimization 
passes to perform)

Note: I realize there must be a way to break everything up into stages and use opt at some point and pass the optimization command line options to it, but I've got a pretty large set of Makefiles which would need to change to do that.

There is no way that I know of (but modifying the PassManagerBuilder and rebuilding clang). But you may start by bisecting the input files compiling half of them with O1 and the other half with O0 and recurse till you find the file that is miscompiled with O1. Then you can use opt to test variant of the pipeline on this file and relink manually.

— 

Mehdi

[David Chisnall via llvm-dev]

On 2 Dec 2015, at 17:56, Phil Tomson via llvm-dev <llvm...@lists.llvm.org> wrote:
>
> Now my intent is to figure out which optimization pass is causing the problem by trying each one until I hit the same problem as with -O1, but clang itself doesn't seem to allow these commandline options, for example:
>
> clang -loops -lcssa ....
>
>

If you want to pass LLVM arguments to clang, then you must prefix them with -llvm (e.g. -mllvm -loops -mllvm lcssa).

David

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Phil Tomson via llvm-dev]

On Wed, Dec 2, 2015 at 10:17 AM, David Chisnall <David.C...@cl.cam.ac.uk> wrote:

On 2 Dec 2015, at 17:56, Phil Tomson via llvm-dev <llvm...@lists.llvm.org> wrote:
>
> Now my intent is to figure out which optimization pass is causing the problem by trying each one until I hit the same problem as with -O1, but clang itself doesn't seem to allow these commandline options, for example:
>
> clang -loops -lcssa ....
>
>

If you want to pass LLVM arguments to clang, then you must prefix them with -llvm (e.g. -mllvm -loops -mllvm lcssa).

I just tried this:
CFLAGS += -mllvm adce -mllvm loops -mllvm loop-simplify -mllvm lcssa -mllvm simplify-libcalls

And get:

clang (LLVM option parsing): Unknown command line argument 'adce'.  Try: 'clang (LLVM option parsing) -help'
clang (LLVM option parsing): Unknown command line argument 'loops'.  Try: 'clang (LLVM option parsing) -help'
clang (LLVM option parsing): Unknown command line argument 'loop-simplify'.  Try: 'clang (LLVM option parsing) -help'
clang (LLVM option parsing): Unknown command line argument 'lcssa'.  Try: 'clang (LLVM option parsing) -help'
clang (LLVM option parsing): Unknown command line argument 'simplify-libcalls'.  Try: 'clang (LLVM option parsing) -help'

 

David

[David Chisnall via llvm-dev]

On 2 Dec 2015, at 18:30, Phil Tomson <phil.a...@gmail.com> wrote:
>
>> If you want to pass LLVM arguments to clang, then you must prefix them with -llvm (e.g. -mllvm -loops -mllvm lcssa).
>
>
> I just tried this:
> CFLAGS += -mllvm adce -mllvm loops -mllvm loop-simplify -mllvm lcssa -mllvm simplify-libcalls
>
> And get:
>
> clang (LLVM option parsing): Unknown command line argument 'adce'. Try: 'clang (LLVM option parsing) -help'
> clang (LLVM option parsing): Unknown command line argument 'loops'. Try: 'clang (LLVM option parsing) -help'
> clang (LLVM option parsing): Unknown command line argument 'loop-simplify'. Try: 'clang (LLVM option parsing) -help'
> clang (LLVM option parsing): Unknown command line argument 'lcssa'. Try: 'clang (LLVM option parsing) -help'
> clang (LLVM option parsing): Unknown command line argument 'simplify-libcalls'. Try: 'clang (LLVM option parsing) -help'

Sorry, I wasn’t thinking. These are options provided by opt, not by LLVM libraries. Clang has made an explicit design decision not to expose the optimisation pipeline details to consumers (because it’s far from stable and there is no desire to maintain backwards compatibility for things that are implementation details and because it’s very hard for someone to use correctly). Your best bet if you *really* want this is probably to wrap clang in a shell script that does something along the lines of clang -c -emit-llvm && opt -whatever | llc, with appropriate handling / replacing of the last -o argument to clang.

[Bill Seurer via llvm-dev]

On 12/02/15 11:56, Phil Tomson via llvm-dev wrote:
> Note: I realize there must be a way to break everything up into stages

> and use /opt/ at some point and pass the optimization command line

> options to it, but I've got a pretty large set of Makefiles which would
> need to change to do that.

Using opt is the way to go.

For your Makefile issue write your own "clang" script that is ahead of
the real clang. Have it run clang/opt/llc for you and then you can
easily change the optimization phases for opt.
--

-Bill Seurer

[Phil Tomson via llvm-dev]

I have a feeling this would get rather tricky given that there are several libraries linked in to the final executable as well. It would have to look something like:

clang -c file.c -emit-llvm > opt -... > llc ... > as > ld (libs to link in)

Someone above mentioned modifying PassManagerBuilder and rebuilding clang - I'd guess there's a list of optimization passes performed somewhere that PassManagerBuilder references, but I haven't been in the optimization pass area of LLVM much yet. Any pointers on that approach?

Phil

 

David

[Mehdi Amini via llvm-dev]

Fairly trivial, have a look at http://llvm.org/docs/doxygen/html/PassManagerBuilder_8cpp_source.html#l00186 and http://llvm.org/docs/doxygen/html/PassManagerBuilder_8cpp_source.html#l00165

If could can see how “OptLevel” impacts what is done.

— 

Mehdi

[David Chisnall via llvm-dev]

On 2 Dec 2015, at 19:00, Phil Tomson <phil.a...@gmail.com> wrote:
>
> I have a feeling this would get rather tricky given that there are several libraries linked in to the final executable as well. It would have to look something like:
>
> clang -c file.c -emit-llvm > opt -... > llc ... > as > ld (libs to link in)

You don’t need as (llc can emit .o files). Hopefully, your build system has different CC / CXX / LD values, so the final link is separate too.

> Someone above mentioned modifying PassManagerBuilder and rebuilding clang - I'd guess there's a list of optimization passes performed somewhere that PassManagerBuilder references, but I haven't been in the optimization pass area of LLVM much yet. Any pointers on that approach?

You can take a look in BackendUtil.cpp for how the passes are constructed for clang. Note that it is quite complex and depends a lot on your language options...

[serge guelton via llvm-dev]

On Wed, Dec 02, 2015 at 12:42:44PM -0600, Bill Seurer via llvm-dev wrote:
> On 12/02/15 11:56, Phil Tomson via llvm-dev wrote:
> >Note: I realize there must be a way to break everything up into stages
> >and use /opt/ at some point and pass the optimization command line
> >options to it, but I've got a pretty large set of Makefiles which would
> >need to change to do that.
>
> Using opt is the way to go.
>
> For your Makefile issue write your own "clang" script that is ahead of the
> real clang. Have it run clang/opt/llc for you and then you can easily
> change the optimization phases for opt.

If you're planning to call your own passes, you can also register them
for clang use, as described in the tutorial we gave at LLVM DEV 2015.
See for instance:

https://github.com/quarkslab/llvm-dev-meeting-tutorial-2015/blob/master/MBA/MBA.cpp#L157

[Phil Tomson via llvm-dev]

On Wed, Dec 2, 2015 at 11:06 AM, David Chisnall <David.C...@cl.cam.ac.uk> wrote:

On 2 Dec 2015, at 19:00, Phil Tomson <phil.a...@gmail.com> wrote:
>
> I have a feeling this would get rather tricky given that there are several libraries linked in to the final executable as well. It would have to look something like:
>
> clang -c file.c -emit-llvm > opt -... > llc ... > as > ld (libs to link in)

You don’t need as (llc can emit .o files). 

How do you get llc to emit a .o file (I seem to only be able to get it to output a .s file)?

[Mehdi Amini via llvm-dev]

On Dec 4, 2015, at 10:18 AM, Phil Tomson via llvm-dev <llvm...@lists.llvm.org> wrote:

On Wed, Dec 2, 2015 at 11:06 AM, David Chisnall <David.C...@cl.cam.ac.uk> wrote:

On 2 Dec 2015, at 19:00, Phil Tomson <phil.a...@gmail.com> wrote:
>
> I have a feeling this would get rather tricky given that there are several libraries linked in to the final executable as well. It would have to look something like:
>
> clang -c file.c -emit-llvm > opt -... > llc ... > as > ld (libs to link in)

You don’t need as (llc can emit .o files). 

How do you get llc to emit a .o file (I seem to only be able to get it to output a .s file)?

$ llc --help | grep filetype -A 3
  -filetype                         - Choose a file type (not all types are supported by all targets):
    =asm                            -   Emit an assembly ('.s') file
    =obj                            -   Emit a native object ('.o') file
    =null                           -   Emit nothing, for performance testing

— 

Mehdi

 

Hopefully, your build system has different CC / CXX / LD values, so the final link is separate too.

 

> Someone above mentioned modifying PassManagerBuilder and rebuilding clang - I'd guess there's a list of optimization passes performed somewhere that PassManagerBuilder references, but I haven't been in the optimization pass area of LLVM much yet. Any pointers on that approach?

You can take a look in BackendUtil.cpp for how the passes are constructed for clang.  Note that it is quite complex and depends a lot on your language options...

David

[Bill Seurer via llvm-dev]

On 12/04/15 12:18, Phil Tomson via llvm-dev wrote:
> How do you get llc to emit a .o file (I seem to only be able to get it
> to output a .s file)?

Here are some notes from the last time I did it that might be helpful.
I was removing optimization passes looking for what was causing a problem.

clang -O2 -mllvm -disable-llvm-optzns -emit-llvm -c mysource.c -o
mysource.llvm.bc

opt ...specify_passes_here... mysource.llvm.bc -o mysource.llvm.ll

llc mysource.llvm.ll -filetype=obj -o mysource.llvm.o

To see all the passes opt would normally use:

opt -O2 -disable-output -debug-pass=Arguments mysource.llvm.bc
--

-Bill Seurer