[LLVMdev] RFC - Stop ignoring -fprofile-generate and -fprofile-use

[Diego Novillo]

The flags -fprofile-generate and -fprofile-use are currently ignored for GCC compatibility.  I would like to enable them and give them similar semantics to GCC.  These flags are baked pretty deeply into our build environment, so supporting them at the driver level will make our lives a lot simpler.

From https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html:

-fprofile-generate

-fprofile-generate=path

Enable options usually used for instrumenting application to produce profile useful for later recompilation with profile feedback based optimization. You must use -fprofile-generate both when compiling and when linking your program.

The following options are enabled: -fprofile-arcs, -fprofile-values, -fvpt.

If path is specified, GCC looks at the path to find the profile feedback data files. See -fprofile-dir. 

-fprofile-use

-fprofile-use=path

Enable profile feedback-directed optimizations, and the following optimizations which are generally profitable only with profile feedback available: -fbranch-probabilities, -fvpt, -funroll-loops, -fpeel-loops, -ftracer, -ftree-vectorize, and ftree-loop-distribute-patterns.

By default, GCC emits an error message if the feedback profiles do not match the source code. This error can be turned into a warning by using -Wcoverage-mismatch. Note this may result in poorly optimized code.

If path is specified, GCC looks at the path to find the profile feedback data files. See -fprofile-dir. 

Note that the argument to -fprofile-generate and -fprofile-use is not a file name. It is a path prefix used to store the generated profile (in GCC's case, each object file in the binary generates its own profile file). In Clang, the flags would do the following:

1. -fprofile-generate=path-prefix would cause the instrumented binary to write the profile <path-prefix>/default.profraw. If <path-prefix> does not exist, it is created by the runtime.
2. -fprofile-use=path-prefix would cause the compiler to read from <path-prefix>/default.profile.
3. I could also add support for -fprofile-dir, but we don't really use it internally.

As with -fprofile-instr-generate, LLVM_PROFILE_FILE would override the path prefix and name of the profile file.

Does this sound reasonable?

Thanks.  Diego.

[Duncan P. N. Exon Smith]

+cfe-dev

_______________________________________________
LLVM Developers mailing list
LLV...@cs.uiuc.edu http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

[Justin Bogner]

On 2015 Jun 17, at 13:53, Diego Novillo <dnov...@google.com> wrote:
> The flags -fprofile-generate and -fprofile-use are currently ignored
> for GCC compatibility. I would like to enable them and give them
> similar semantics to GCC. These flags are baked pretty deeply into
> our build environment, so supporting them at the driver level will
> make our lives a lot simpler.

This seems like a fairly reasonable idea. We'll have to make sure it's
clear (in the docs or whatever) that you can't use clang to generate a
profile and gcc to consume it and vice versa, but that doesn't seem like
a big deal.

The path-prefix versions are a bit of an odd fit for the instrprof
stuff - I'd prefer if they dealt with the path to the file. I can see
how that might be difficult to do and still be compatible though.

Maybe we could make the -fprofile-use= version accept a file or
directory, and if a directory is specified then look for the default?

The other thing that would be nice would be if -fprofile-use=path could
autodetect which kind of profile the path is - that is, It'd be nice to
accept both instrprof and sample profiles with this option. I'd limit
this to the path-to-a-file case though, I think it's too magical when
pointing at a directory.

> • I could also add support for -fprofile-dir, but we don't
> really use it internally.

I don't think it's necessary unless someone actually wants to use it.

[Xinliang David Li]

> The path-prefix versions are a bit of an odd fit for the instrprof
> stuff - I'd prefer if they dealt with the path to the file. I can see
> how that might be difficult to do and still be compatible though.
>
> Maybe we could make the -fprofile-use= version accept a file or
> directory, and if a directory is specified then look for the default?

That sounds reasonable.

>
> The other thing that would be nice would be if -fprofile-use=path could
> autodetect which kind of profile the path is - that is, It'd be nice to
> accept both instrprof and sample profiles with this option. I'd limit
> this to the path-to-a-file case though, I think it's too magical when
> pointing at a directory.

Agree.

David

[Diego Novillo]

On Wed, Jun 17, 2015 at 5:34 PM, Justin Bogner <ma...@justinbogner.com> wrote:

On 2015 Jun 17, at 13:53, Diego Novillo <dnov...@google.com> wrote:

>   • -fprofile-generate=path-prefix would cause the instrumented
>     binary to write the profile <path-prefix>/default.profraw. If
>     <path-prefix> does not exist, it is created by the runtime.
>   • -fprofile-use=path-prefix would cause the compiler to read
>     from <path-prefix>/default.profile.

The path-prefix versions are a bit of an odd fit for the instrprof
stuff - I'd prefer if they dealt with the path to the file. I can see
how that might be difficult to do and still be compatible though.

Maybe we could make the -fprofile-use= version accept a file or
directory, and if a directory is specified then look for the default?

Sure. Sounds reasonable.

 

The other thing that would be nice would be if -fprofile-use=path could
autodetect which kind of profile the path is - that is, It'd be nice to
accept both instrprof and sample profiles with this option.

Sure.  But this may have a wrinkle. The problem here is that GCC already has a different option for sample profiles (-fauto-profile). Someone using the same flags for both compilers (or migrating off of GCC) might find this problematic. Although, I can't really think a scenario where this would happen.

In a sense, anything that is a superset of GCC's current behaviour should be fine. So, I suppose this is OK.

 

I'd limit
this to the path-to-a-file case though, I think it's too magical when
pointing at a directory.

Agreed.

Diego.

[Xinliang David Li]

yes -- you are not removing the old behavior.

>
>>
>> I'd limit
>> this to the path-to-a-file case though, I think it's too magical when
>> pointing at a directory.
>
>
> Agreed.
>
>
> Diego.

_______________________________________________

[Justin Bogner]

Diego Novillo <dnov...@google.com> writes:
> On Wed, Jun 17, 2015 at 5:34 PM, Justin Bogner <ma...@justinbogner.com> wrote:
>> On 2015 Jun 17, at 13:53, Diego Novillo <dnov...@google.com> wrote:
>>> • -fprofile-generate=path-prefix would cause the instrumented
>>> binary to write the profile <path-prefix>/default.profraw. If
>>> <path-prefix> does not exist, it is created by the runtime.
>>> • -fprofile-use=path-prefix would cause the compiler to read
>>> from <path-prefix>/default.profile.
>>
>> The path-prefix versions are a bit of an odd fit for the instrprof
>> stuff - I'd prefer if they dealt with the path to the file. I can see
>> how that might be difficult to do and still be compatible though.
>>
>> Maybe we could make the -fprofile-use= version accept a file or
>> directory, and if a directory is specified then look for the default?
>
> Sure. Sounds reasonable.
>
>> The other thing that would be nice would be if -fprofile-use=path could
>> autodetect which kind of profile the path is - that is, It'd be nice to
>> accept both instrprof and sample profiles with this option.
>
> Sure. But this may have a wrinkle. The problem here is that GCC already has a
> different option for sample profiles (-fauto-profile). Someone using the same
> flags for both compilers (or migrating off of GCC) might find this
> problematic. Although, I can't really think a scenario where this would
> happen.

Convince gcc to do this too? ;)

I'm not terribly attached to this idea if you think it's a bad precedent
to set or something, but it does seem like it'd be convenient enough to
be worthwhile.

> In a sense, anything that is a superset of GCC's current behaviour should be
> fine. So, I suppose this is OK.

_______________________________________________