[LLVMdev] strict aliasing and LLVM

[Xinliang David Li]

Is type based aliasing implemented in LLVM? Simple test case shows it is not, or an option similar to -fstrict-aliasing is needed?

Thanks,

David

[Rafael Espíndola]

On 27 October 2010 18:17, Xinliang David Li <xinli...@gmail.com> wrote:
> Is type based aliasing implemented in LLVM? Simple test case shows it is
> not, or an option similar to -fstrict-aliasing is needed?

It is being implemented. You have to use clang, not llvm-gcc. The
types are encoded as metadata. Compile with -OX and and look for the
TBAA metadata in the .ll file.

> Thanks,
> David

Cheers,
Rafael
_______________________________________________
LLVM Developers mailing list
LLV...@cs.uiuc.edu http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

[Xinliang David Li]

Thanks. Just built clang and saw the meta data and annotations on the memory accesses --  is any opt pass consuming the information?

By the way the build instruction in this page http://clang.llvm.org/get_started.html needs to be updated -- it recommends config (with default settings) and build llvm in the source dir -- it leaves some 'sticky' generated files in the source dir leading to building problems.

David

2010/10/27 Rafael Espíndola <rafael.e...@gmail.com>

[Rafael Espíndola]

2010/10/27 Xinliang David Li <xinli...@gmail.com>:

> Thanks. Just built clang and saw the meta data and annotations on the memory
> accesses --  is any opt pass consuming the information?

The tests in test/Analysis/TypeBasedAliasAnalysis suggest that at
least licm is using it. Also note that
lib/Analysis/TypeBasedAliasAnalysis.cpp defines as enable-tbaa option
that is off by default.

> By the way the build instruction in this
> page http://clang.llvm.org/get_started.html needs to be updated -- it
> recommends config (with default settings) and build llvm in the source dir
> -- it leaves some 'sticky' generated files in the source dir leading to
> building problems.

Thanks. Will try to improve it a bit.

>
> David
> 2010/10/27 Rafael Espíndola <rafael.e...@gmail.com>

Cheers,

[Xinliang David Li]

2010/10/27 Rafael Espíndola <rafael.e...@gmail.com>

2010/10/27 Xinliang David Li <xinli...@gmail.com>:

> Thanks. Just built clang and saw the meta data and annotations on the memory
> accesses --  is any opt pass consuming the information?

The tests in test/Analysis/TypeBasedAliasAnalysis suggest that at
least licm is using it. Also note that
lib/Analysis/TypeBasedAliasAnalysis.cpp defines as enable-tbaa option
that is off by default.

I tried the option -- no much differences in the generated code.

A related question: how to pass the llvm specific options from clang driver?  It supports -Wl, -Wa, -Xanalyzer etc, but there is no documentation on how to pass -enable-tbaa to opt driver. 

llvmc supports -Wo, option, but the option specified after -Wo, seems to be dropped.  Another thing, when using -clang option to llvmc, I got link errors even when -c or -S is specified:

clang -x c  foo.c -emit-llvm-bc -o /tmp/llvm_JnS1o8/foo.bc 

(.text+0x20): undefined reference to `main'

collect2: ld returned 1 exit status

clang: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)

David

[Dan Gohman]

On Oct 28, 2010, at 2:43 PM, Xinliang David Li wrote:

>
>
> 2010/10/27 Rafael Espíndola <rafael.e...@gmail.com>
> 2010/10/27 Xinliang David Li <xinli...@gmail.com>:
> > Thanks. Just built clang and saw the meta data and annotations on the memory
> > accesses -- is any opt pass consuming the information?
>
> The tests in test/Analysis/TypeBasedAliasAnalysis suggest that at
> least licm is using it. Also note that
> lib/Analysis/TypeBasedAliasAnalysis.cpp defines as enable-tbaa option
> that is off by default.

LICM, GVN, and DSE are the major consumers right now. That said, the
current TBAA implementation is not very advanced yet.

> I tried the option -- no much differences in the generated code.

Can you give an example of code you'd expect to be optimized which isn't?

Dan

[Xinliang David Li]

As simple as 

void foo (int n, double *p, int *q)

{

   for (int i = 0; i < n; i++)

     *p += *q;

}

clang -O2 -fstrict-aliasing -emit-llvm -o foo.bc -c foo.c

llc -enable-tbaa -O2 -filetype=asm -o foo.s foo.bc

Memory accesses remain in the loop.

The following works fine:

void foo(int n, double *restrict p, int * restrict *q)

{ 

  ...
}

By the way, Is there a performance category in the llvm bug database?

Thanks,

David

[Nick Lewycky]

Xinliang David Li wrote:
> As simple as
>
> void foo (int n, double *p, int *q)
> {
> for (int i = 0; i < n; i++)
> *p += *q;
> }
>
> clang -O2 -fstrict-aliasing -emit-llvm -o foo.bc -c foo.c
> llc -enable-tbaa -O2 -filetype=asm -o foo.s foo.bc

There's a couple things interacting here:
* clang -fstrict-aliasing -O2 does generate the TBAA info, but it runs
the optimizers without enabling the -enable-tbaa flag, so the optimizers
never look at it. Oops.
* clang -fstrict-aliasing -O0 does *not* generate the TBAA info in the
resulting .bc file. This is probably intended to speed up -O0 builds
even if -fstrict-aliasing is set, but is annoying for debugging what's
going on under the hood.
* If clang -O2 worked by running 'opt' and 'llc' under the hood, we
could tell it to pass a flag along to them, but it doesn't. As it
stands, you can't turn -enable-tbaa on when running clang.

So, putting that together, one way to do it is:

clang -O2 -fstrict-aliasing foo.c -flto -c -o foo.bc
opt -O2 -enable-tbaa foo.bc foo2.bc
llc -O2 -enable-tbaa foo2.bc -o foo2.s

at which point the opt run will hoist the loads into a loop preheader.
Sadly this runs the LLVM optimizers twice (once in clang -O2 and once in
opt) which could skew results.

I think the right thing to do is to teach the clang driver to remove
-fstrict-aliasing from the cc1 invocation when optimizations are off.
This would let us force the flag through with "-Xclang -fstrict-aliasing".

> Memory accesses remain in the loop.
>
> The following works fine:
>
> void foo(int n, double *restrict p, int * restrict *q)
> {
> ...
> }
>
> By the way, Is there a performance category in the llvm bug database?

Nope, we file bugs based on the type of optimization ought to solve it
(ie., there's a Scalar optimizations category, a Loop optimizer
category, Backend: X86, etc.). Many miscellaneous performance
improvements actually live in lib/Target/README.txt (and subdirs of
there) instead of the bug tracker.

Nick

> Thanks,
>
> David
>
> On Thu, Oct 28, 2010 at 5:59 PM, Dan Gohman <goh...@apple.com
> <mailto:goh...@apple.com>> wrote:
>
>
> On Oct 28, 2010, at 2:43 PM, Xinliang David Li wrote:
>
> >
> >
> > 2010/10/27 Rafael Espíndola <rafael.e...@gmail.com

> <mailto:rafael.e...@gmail.com>>

> > 2010/10/27 Xinliang David Li <xinli...@gmail.com

> <mailto:xinli...@gmail.com>>:

> > > Thanks. Just built clang and saw the meta data and annotations
> on the memory
> > > accesses -- is any opt pass consuming the information?
> >
> > The tests in test/Analysis/TypeBasedAliasAnalysis suggest that at
> > least licm is using it. Also note that
> > lib/Analysis/TypeBasedAliasAnalysis.cpp defines as enable-tbaa option
> > that is off by default.
>
> LICM, GVN, and DSE are the major consumers right now. That said, the
> current TBAA implementation is not very advanced yet.
>
> > I tried the option -- no much differences in the generated code.
>
> Can you give an example of code you'd expect to be optimized which
> isn't?
>
> Dan
>
>
>
>

[Benjamin Kramer]

On 29.10.2010, at 09:26, Nick Lewycky wrote:

> * If clang -O2 worked by running 'opt' and 'llc' under the hood, we
> could tell it to pass a flag along to them, but it doesn't. As it
> stands, you can't turn -enable-tbaa on when running clang.
>
> So, putting that together, one way to do it is:
>
> clang -O2 -fstrict-aliasing foo.c -flto -c -o foo.bc
> opt -O2 -enable-tbaa foo.bc foo2.bc
> llc -O2 -enable-tbaa foo2.bc -o foo2.s

clang -O2 foo.c -S -o foo.s -mllvm -enable-tbaa

[Dale Johannesen]

> * clang -fstrict-aliasing -O0 does *not* generate the TBAA info in the
> resulting .bc file. This is probably intended to speed up -O0 builds
> even if -fstrict-aliasing is set, but is annoying for debugging what's
> going on under the hood.

I would expect -O0 to turn off strict-aliasing, so this seems like correct behavior, applying the usual "last flag wins" rule for conflicts. -O0 -fstrict-aliasing should do what you want.

[Dan Gohman]

On Oct 29, 2010, at 12:26 AM, Nick Lewycky wrote:

> Xinliang David Li wrote:
>> As simple as
>>
>> void foo (int n, double *p, int *q)
>> {
>> for (int i = 0; i < n; i++)
>> *p += *q;
>> }
>>
>> clang -O2 -fstrict-aliasing -emit-llvm -o foo.bc -c foo.c
>> llc -enable-tbaa -O2 -filetype=asm -o foo.s foo.bc
>
> There's a couple things interacting here:
> * clang -fstrict-aliasing -O2 does generate the TBAA info, but it runs the optimizers without enabling the -enable-tbaa flag, so the optimizers never look at it. Oops.
> * clang -fstrict-aliasing -O0 does *not* generate the TBAA info in the resulting .bc file. This is probably intended to speed up -O0 builds even if -fstrict-aliasing is set, but is annoying for debugging what's going on under the hood.
> * If clang -O2 worked by running 'opt' and 'llc' under the hood, we could tell it to pass a flag along to them, but it doesn't. As it stands, you can't turn -enable-tbaa on when running clang.

In case there is any confusion, the -enable-tbaa option is temporary. TBAA is
a new feature which is still under development.

[Xinliang David Li]

On Fri, Oct 29, 2010 at 12:26 AM, Nick Lewycky <nich...@mxc.ca> wrote:

Xinliang David Li wrote:

As simple as

void foo (int n, double *p, int *q)
{
   for (int i = 0; i < n; i++)
     *p += *q;
}

clang -O2 -fstrict-aliasing -emit-llvm -o foo.bc -c foo.c
llc -enable-tbaa -O2 -filetype=asm -o foo.s foo.bc

There's a couple things interacting here:
 * clang -fstrict-aliasing -O2 does generate the TBAA info, but it runs the optimizers without enabling the -enable-tbaa flag, so the optimizers never look at it. Oops.
 * clang -fstrict-aliasing -O0 does *not* generate the TBAA info in the resulting .bc file. This is probably intended to speed up -O0 builds even if -fstrict-aliasing is set, but is annoying for debugging what's going on under the hood.
 * If clang -O2 worked by running 'opt' and 'llc' under the hood, we could tell it to pass a flag along to them, but it doesn't. As it stands, you can't turn -enable-tbaa on when running clang.

So, putting that together, one way to do it is:

 clang -O2 -fstrict-aliasing foo.c -flto -c -o foo.bc
 opt -O2 -enable-tbaa foo.bc foo2.bc

   -o foo2.bc

 

 llc -O2 -enable-tbaa foo2.bc -o foo2.s

at which point the opt run will hoist the loads into a loop preheader. Sadly this runs the LLVM optimizers twice (once in clang -O2 and once in opt) which could skew results.

Yes, I verified these steps work, but my head is spinning:

1) does -flto has the same effect as -emit-llvm ? FE emits llvm bitcode and exit without invoking llvm backend?

2) why do you need to invoke both opt and llc -- I verified invoking just llc is also fine.

3) more general question -- is opt just a barebone llc without invoking any llvm passes? So why is there a need for two opt driver?

Thanks,

David

[Xinliang David Li]

Thanks for the -m tip to pass llvm options.  Why is it not documented anywhere. See http://clang.llvm.org/docs/UsersManual.html#commandline

David

[Chris Lattner]

On Oct 29, 2010, at 11:35 AM, Xinliang David Li wrote:

Thanks for the -m tip to pass llvm options.  Why is it not documented anywhere. See http://clang.llvm.org/docs/UsersManual.html#commandline

-mllvm flags are somewhat equivalent to gcc -param options.  -mllvm flags are for compiler hackers to play with, and are not stable or documented.  Once TBAA is stable and reliable it will be controlled with -fstrict-aliasing as you'd expect.

-Chris

[Xinliang David Li]

On Fri, Oct 29, 2010 at 3:54 PM, Chris Lattner <clat...@apple.com> wrote:

On Oct 29, 2010, at 11:35 AM, Xinliang David Li wrote:

Thanks for the -m tip to pass llvm options.  Why is it not documented anywhere. See http://clang.llvm.org/docs/UsersManual.html#commandline

-mllvm flags are somewhat equivalent to gcc -param options.  -mllvm flags are for compiler hackers to play with, and are not stable or documented.  Once TBAA is stable and reliable it will be controlled with -fstrict-aliasing as you'd expect.

-mllvm is also useful for passing options like -print-after=xxx.  May be it should be mentioned when invoking 'clang --help-hidden'.

Thanks,

David

[Chris Lattner]

On Oct 29, 2010, at 4:08 PM, Xinliang David Li wrote:

On Fri, Oct 29, 2010 at 3:54 PM, Chris Lattner <clat...@apple.com> wrote:

On Oct 29, 2010, at 11:35 AM, Xinliang David Li wrote:

Thanks for the -m tip to pass llvm options.  Why is it not documented anywhere. See http://clang.llvm.org/docs/UsersManual.html#commandline

-mllvm flags are somewhat equivalent to gcc -param options.  -mllvm flags are for compiler hackers to play with, and are not stable or documented.  Once TBAA is stable and reliable it will be controlled with -fstrict-aliasing as you'd expect.

-mllvm is also useful for passing options like -print-after=xxx.  May be it should be mentioned when invoking 'clang --help-hidden'.

But that's another example of something for compiler hackers, which isn't stable or documented :-)

-Chris

[Rafael Espíndola]

> Yes, I verified these steps work, but my head is spinning:
> 1) does -flto has the same effect as -emit-llvm ? FE emits llvm bitcode and
> exit without invoking llvm backend?

I think they are the same, but maybe -flto also affects which passes
are run. Not sure. I always used -emit-llvm...

> 2) why do you need to invoke both opt and llc -- I verified invoking just
> llc is also fine.
> 3) more general question -- is opt just a barebone llc without invoking any
> llvm passes? So why is there a need for two opt driver?

opt contains the IL -> IL optimizations. It is what you want to use
for testing that a loop is unrolled for example.
llc does the IL -> .s (or .o) transformation. It will also run low level passes.

I guess they could be merged. Never tough about the trade offs.

> Thanks,
> David
>

Cheers,
Rafael

[Rafael Espíndola]

> clang -x c  foo.c -emit-llvm-bc -o /tmp/llvm_JnS1o8/foo.bc
> (.text+0x20): undefined reference to `main'
> collect2: ld returned 1 exit status
> clang: error: linker (via gcc) command failed with exit code 1 (use -v to
> see invocation)

Without a -c I think clang is trying to link the llvm IL file. For
that to work you would need a linker that understands LLVM IL. Both
the apple linker and gold support plugins for doing it.

> David

[Xinliang David Li]

2010/10/29 Rafael Espíndola <rafael.e...@gmail.com>

> clang -x c  foo.c -emit-llvm-bc -o /tmp/llvm_JnS1o8/foo.bc
> (.text+0x20): undefined reference to `main'
> collect2: ld returned 1 exit status
> clang: error: linker (via gcc) command failed with exit code 1 (use -v to
> see invocation)

Without a -c I think clang is trying to link the llvm IL file. For
that to work you would need a linker that understands LLVM IL. Both
the apple linker and gold support plugins for doing it.

Then it looks like a bug in llvmc driver -- -c is not passed to clang.

David

 

> David

Cheers,
Rafael

[Chris Lattner]

I strongly recommend not using llvmc unless you know exactly what you're doing and what the features and limitations of llvmc are.  Please use the clang driver.

-Chris

[Xinliang David Li]

Is llvmc just a wrapper on top of  llvm-gcc and clang?  In search of a way to pass down internal llvm options ( I did not know about -m yet), I found that llvmc has -Wo, option (which does not work), that is why I tried it.  So many different driver programs makes it little confusing to newbies.

Thanks,

David

 

-Chris

[Nick Lewycky]

Dan Gohman wrote:
>
> On Oct 29, 2010, at 12:26 AM, Nick Lewycky wrote:
>
>> Xinliang David Li wrote:
>>> As simple as
>>>
>>> void foo (int n, double *p, int *q)
>>> {
>>> for (int i = 0; i< n; i++)
>>> *p += *q;
>>> }
>>>
>>> clang -O2 -fstrict-aliasing -emit-llvm -o foo.bc -c foo.c
>>> llc -enable-tbaa -O2 -filetype=asm -o foo.s foo.bc
>>
>> There's a couple things interacting here:
>> * clang -fstrict-aliasing -O2 does generate the TBAA info, but it runs the optimizers without enabling the -enable-tbaa flag, so the optimizers never look at it. Oops.
>> * clang -fstrict-aliasing -O0 does *not* generate the TBAA info in the resulting .bc file. This is probably intended to speed up -O0 builds even if -fstrict-aliasing is set, but is annoying for debugging what's going on under the hood.
>> * If clang -O2 worked by running 'opt' and 'llc' under the hood, we could tell it to pass a flag along to them, but it doesn't. As it stands, you can't turn -enable-tbaa on when running clang.
>
> In case there is any confusion, the -enable-tbaa option is temporary. TBAA is
> a new feature which is still under development.

That's fine, but we'll need a way to get bitcode with TBAA metadata but
without any optimizations having been run. (Maybe -O2 -fstrict-aliasing
-mllvm -disable-llvm-optzns?)

Nick

[Nick Lewycky]

Yes, -flto and -emit-llvm are synonyms.

> 2) why do you need to invoke both opt and llc -- I verified invoking
> just llc is also fine.

"llc" is really just codegen; the only optimizations it does are ones
that are naturally part of lowering from llvm IR to assembly. For
example, that includes another run of loop invariant code motion because
some loads may have been added -- such as a load of the GOT pointer --
which weren't there in the IR to be hoisted.

"opt" runs any IR pass. You can ask run a single optimization, for
example "opt -licm" or you can run an analysis pass like scalar
evolutions with "opt -analyze -scalar-evolution". This is where the bulk
of LLVM's optimizations live.

> 3) more general question -- is opt just a barebone llc without invoking
> any llvm passes? So why is there a need for two opt driver?

I think of it as opt transforms .bc -> .bc and llc transforms .bc -> .s.

Nick

> Thanks,
>
> David
>
>
> I think the right thing to do is to teach the clang driver to remove
> -fstrict-aliasing from the cc1 invocation when optimizations are
> off. This would let us force the flag through with "-Xclang
> -fstrict-aliasing".
>
>
> Memory accesses remain in the loop.
>
> The following works fine:
>
> void foo(int n, double *restrict p, int * restrict *q)
> {
> ...
> }
>
> By the way, Is there a performance category in the llvm bug
> database?
>
>
> Nope, we file bugs based on the type of optimization ought to solve
> it (ie., there's a Scalar optimizations category, a Loop optimizer
> category, Backend: X86, etc.). Many miscellaneous performance
> improvements actually live in lib/Target/README.txt (and subdirs of
> there) instead of the bug tracker.
>
> Nick
>
> Thanks,
>
> David
>
> On Thu, Oct 28, 2010 at 5:59 PM, Dan Gohman <goh...@apple.com
> <mailto:goh...@apple.com>

> <mailto:goh...@apple.com <mailto:goh...@apple.com>>> wrote:
>
>
> On Oct 28, 2010, at 2:43 PM, Xinliang David Li wrote:
>
> >
> >
> > 2010/10/27 Rafael Espíndola <rafael.e...@gmail.com
> <mailto:rafael.e...@gmail.com>

> <mailto:rafael.e...@gmail.com

> <mailto:rafael.e...@gmail.com>>>
>
> > 2010/10/27 Xinliang David Li <xinli...@gmail.com
> <mailto:xinli...@gmail.com>

> <mailto:xinli...@gmail.com <mailto:xinli...@gmail.com>>>:

>
> > > Thanks. Just built clang and saw the meta data and annotations
> on the memory
> > > accesses -- is any opt pass consuming the information?
> >
> > The tests in test/Analysis/TypeBasedAliasAnalysis suggest that at
> > least licm is using it. Also note that
> > lib/Analysis/TypeBasedAliasAnalysis.cpp defines as
> enable-tbaa option
> > that is off by default.
>
> LICM, GVN, and DSE are the major consumers right now. That
> said, the
> current TBAA implementation is not very advanced yet.
>
> > I tried the option -- no much differences in the generated code.
>
> Can you give an example of code you'd expect to be optimized
> which
> isn't?
>
> Dan
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list

> LLV...@cs.uiuc.edu <mailto:LLV...@cs.uiuc.edu>

[Dan Gohman]

On Oct 30, 2010, at 1:34 AM, Nick Lewycky wrote:
> (Maybe -O2 -fstrict-aliasing -mllvm -disable-llvm-optzns?)

Yep; you can do this today. And the -fstrict-aliasing is actually redundant
since it's on by default with -O2.

Dan

[Chris Lattner]

llvmc is an experimental wrapper, not part of the normal toolchain.  I'm not sure why, but it was used by the PIC16 toolchain (when it was in mainline), it allows defining compiler drivers that don't follow the GCC command line syntax.

I completely agree that llvmc is confusing.  I would certainly agree with its removal from mainline, but it is actively maintained and presumably there is a reason for that :-)

Lets look at this another way: maybe this is a documentation issue.  Did llvmc's existence make you want to use it, or was there some documentation that pushed you in that direction?  If it's a doc issue, it is easy to fix.

Thanks!

-Chris

[Xinliang David Li]

I thought llc include all standard optimization passes (equivalent to opt -std-compile-opts + machine code generation) --- it seems more natural (to me) to have  a single optimization driver that takes llvm bit code as the input.  

 

David 

[Xinliang David Li]

Lack of documentation saying llvmc is experimental, it is actually more natural for me to pick llvmc than llvm-gcc or clang as the compiler driver. LLVM is a great brand name, but it seems more a brand name of a new technology, not a product (unlike gcc which is a both a technology and product).  llvm-gcc seems a bad name - it is more like gcc-llvm (gcc  FE with alternative backend which is llvm backend.  Open64 uses gcc FE, but there is no gcc in their product name.). clang is compiler FE, it can be coupled with gcc backend as well, so using 'clang' as the product name does not sound natural.  Other questions that may pop up user mind is how is clang and llvm-gcc compare in terms of performance, are they configured with the same set of optimization passes?  Without deep investigation, my guess was that 'llvmc'  (maybe it should be called lcc, similar to icc, gcc, aCC, occ, etc) is an effort to unify the compiler drivers and saves users some confusion :). 

Thanks,

David

 

Thanks!

-Chris

[Eric Christopher]

On Oct 30, 2010, at 3:26 PM, Xinliang David Li wrote:

> I thought llc include all standard optimization passes (equivalent to opt -std-compile-opts + machine code generation) --- it seems more natural (to me) to have a single optimization driver that takes llvm bit code as the input.

And that is opt, it handles all machine independent optimizations (though taking advantage of target data if it knows about it). llc handles target dependent code generation as well as some optimization passes that run on machine instrs like the machine licm that Nick was talking about.

-eric

_______________________________________________
LLVM Developers mailing list

LLV...@cs.uiuc.edu http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev