[LLVMdev] Questions about LLVM PGO and autoFDO

[猫妖]

Hi, all

  Recently I'm trying to use LLVM PGO and autoFDO. However I have some problems in the process. 

  LLVM source code is updated on April 9th. Operating system  is SUSE x86_64

  1. Problems in instrumentation based PGO:

  clang -O2 -fprofile-instr-generate test.c -o a.out

  ./a.out     (then default.profraw is generated)

  clang -O2 -fprofile-instr-use=default.profraw test.c -o a.out

  error: pgo data file has malformed function entry

  I found that the run-time function "writeFile" in InstrProfilingFile.cpp didn't write '\n'. 

  But when parsing the profile data,  in CodeGenPGO.cpp line 56, it tried to find '\n' like below:

  CurPtr = strchr(CurPtr, '\n');

  if (!CurPtr) {

    ReportBadPGOData(CGM, "pgo data file has malformed function entry");

    return;

  }

  

  2. Problems in autoFDO: 

  Actually the problem happened in using create_llvm_prof, transformation is failed.

  clang -O2 -g test.c -o a.out

  perf record -b ./a.out    (perf version is 0.0.2, "-b" option is not recognized, why?)

  change to: perf record ./a.out   (so perf.data is generated)

  create_llvm_prof --binary=./a.out --out=code.prof  (it read a.out and perf.data)

  error: Malformed .note.gnu.build-id section

           no build id found in the binary

  it seems that a.out has no build id.

  Someone who can help me?

  Thanks,

  Kaidong Chen

[Diego Novillo]

On Mon, May 12, 2014 at 12:59 PM, 猫妖 <spca...@qq.com> wrote:

> 2. Problems in autoFDO:
> Actually the problem happened in using create_llvm_prof, transformation is
> failed.
> clang -O2 -g test.c -o a.out
> perf record -b ./a.out (perf version is 0.0.2, "-b" option is not
> recognized, why?)
> change to: perf record ./a.out (so perf.data is generated)
> create_llvm_prof --binary=./a.out --out=code.prof (it read a.out and
> perf.data)
> error: Malformed .note.gnu.build-id section
> no build id found in the binary
> it seems that a.out has no build id.

This error message from create_llvm_prof is actually harmless. Dehao
will be removing it at some point. The file code.prof should have
content and clang will be able to read it. Please try:

$ clang -fprofile-sample-use=code.prof -g ...

Diego.

_______________________________________________
LLVM Developers mailing list
LLV...@cs.uiuc.edu http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

[Justin Bogner]

On Monday, May 12, 2014, 猫妖 <spca...@qq.com> wrote:

Hi, all

  Recently I'm trying to use LLVM PGO and autoFDO. However I have some problems in the process. 

  LLVM source code is updated on April 9th. Operating system  is SUSE x86_64

  1. Problems in instrumentation based PGO:

  clang -O2 -fprofile-instr-generate test.c -o a.out

  ./a.out     (then default.profraw is generated)

  clang -O2 -fprofile-instr-use=default.profraw test.c -o a.out

  error: pgo data file has malformed function entry

  I found that the run-time function "writeFile" in InstrProfilingFile.cpp didn't write '\n'. 

  But when parsing the profile data,  in CodeGenPGO.cpp line 56, it tried to find '\n' like below:

  CurPtr = strchr(CurPtr, '\n');

  if (!CurPtr) {

    ReportBadPGOData(CGM, "pgo data file has malformed function entry");

    return;

  }

Sorry that this isn't documented very well. I'm working on some documentation for the instrumentation based profiling that should make this less of a guessing game.

You need to run `llvm-profdata merge` on the raw profile data to generate the format that clang understands. Something like:

  llvm-profdata merge default.profraw -o default.profdata

Then use default.profdata as the input to -fprofile-instr-use.

[猫妖]

On Monday, May 12, 2014, 猫妖 <spca...@qq.com> wrote:

Hi, all

  Recently I'm trying to use LLVM PGO and autoFDO. However I have some problems in the process. 

  LLVM source code is updated on April 9th. Operating system  is SUSE x86_64

  1. Problems in instrumentation based PGO:

  clang -O2 -fprofile-instr-generate test.c -o a.out

  ./a.out     (then default.profraw is generated)

  clang -O2 -fprofile-instr-use=default.profraw test.c -o a.out

  error: pgo data file has malformed function entry

  I found that the run-time function "writeFile" in InstrProfilingFile.cpp didn't write '\n'. 

  But when parsing the profile data,  in CodeGenPGO.cpp line 56, it tried to find '\n' like below:

  CurPtr = strchr(CurPtr, '\n');

  if (!CurPtr) {

    ReportBadPGOData(CGM, "pgo data file has malformed function entry");

    return;

  }

>>Sorry that this isn't documented very well. I'm working on some documentation for the instrumentation 

>>based profiling that should make this less of a guessing game.

>>You need to run `llvm-profdata merge` on the raw profile data to generate the format that clang 

>>understands. Something like:

>>  llvm-profdata merge default.profraw -o default.profdata

>>Then use default.profdata as the input to -fprofile-instr-use.

     It has worked well after I do as you say. Thank you!

     And I want to confirm what passes benefits from profdata-guided feedback currently.

     Is the profiledata feeded back to all of basicblock replacement, function inline, register allocation

     and loop-unroll, or only basicblock replacement ?

     

     I'll test performance improvement of PGO on some benchmarks.

     I think you already have some test results.

     Thanks,

     Kaidong Chen

 

  

[猫妖]

On Tue, May 13, 2014 at 1:15 PM, 猫妖 <spca...@qq.com> wrote:

   2. Problems in autoFDO:
  Actually the problem happened in using create_llvm_prof, transformation is
  failed.
  clang -O2 -g test.c -o a.out
  perf record -b ./a.out    (perf version is 0.0.2, "-b" option is not recognized, why?)

>>Hm, now I noticed the perf version you are using. That's too old. You
>>will need perf and kernel newer than 3.4. I'm currently using
>>3.8.13.13


>>Diego.

  Yes, my perf is so old. It would bring errors like:Read wrong magic,(because

        "PERFFILE" has been changed to "PERFILE2") and others.

And I have updated my perf to about version 3.4, which already support "-b"

        option, but still convert failed.  (I didn't add "-b" option because my linux OS

        is version2.6, lower than 3.4. Meaning no Last Branch Records samples.UserGuide

        said LBR is not prerequisite. I don't know whether it is the reason for convert failure,

        or it is because of different perfdata format from different perf version. )

I will try the newest perf you recommended.