[llvm-dev] PGO, zlib and 'default.profraw'

[ORiordan, Martin via llvm-dev]

We are trying to get PGO working for our embedded out-of-tree target, but the utility ‘llvm-profdata’ does not like the data we are giving it.

 

Because this is not a hosted environment, we have to off-chip the profiling data ourselves, and although the data looks okay, ‘llvm-profdata’ reports the following error:

 

llvm-profdata show -all-functions -counts -detailed-summary -text -static-func-full-module-prefix ~/Downloads/default.profraw

error: ~/Downloads/default.profraw: Failed to uncompress data (zlib)

 

But even for a hosted system, it is not clear to me where ‘zlib’ gets involved in compressing the data.  The start of our data looks like:

 

hexdump -C default.profraw

00000000  81 52 66 6f 72 70 6c ff  04 00 00 00 00 00 00

00  |.Rforpl.........|

00000010  3c 00 00 00 00 00 00 00  28 01 00 00 00 00 00

00  |<.......(.......|

 

Which looks valid for Little-Endian, 32-bit and LLVM v4.0.

 

What do we need to do to prepare this raw data from the device into the form that ‘llvm-profdata’ needs to see?

 

Thanks,

 

            MartinO

 

 

--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

[陳韋任 via llvm-dev]

You might want to set breakpoint on readPGOFuncNameStrings [1] to see
what's going on.

[1] http://llvm.org/doxygen/InstrProf_8cpp_source.html

2017-07-18 22:05 GMT+08:00 ORiordan, Martin via llvm-dev
<llvm...@lists.llvm.org>:

> _______________________________________________
> LLVM Developers mailing list
> llvm...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

--
Wei-Ren Chen (陳韋任)
Homepage: https://people.cs.nctu.edu.tw/~chenwj
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Xinliang David Li via llvm-dev]

The __llvm_prf_names section is compressed but your llvm-profdata tool is probably not built with zlib enabled.  

To disable compression, use option -mllvm -enable-name-compression=false  in your instrumentation build.

David

[Justin Bogner via llvm-dev]

Can we improve the error message here? We should be able to check
zlib::isAvailable and give an error like "profile uses zlib compression
but the profile reader was built without zlib support" or so in this
case.

Xinliang David Li via llvm-dev <llvm...@lists.llvm.org> writes:
> The __llvm_prf_names section is compressed but your llvm-profdata tool is
> probably not built with zlib enabled.
>
> To disable compression, use option -mllvm -enable-name-compression=false
> in your instrumentation build.
>
> David
>
> On Tue, Jul 18, 2017 at 7:05 AM, ORiordan, Martin via llvm-dev <
> llvm...@lists.llvm.org> wrote:
>
>> We are trying to get PGO working for our embedded out-of-tree target, but
>> the utility ‘llvm-profdata’ does not like the data we are giving it.
>>
>>
>>
>> Because this is not a hosted environment, we have to off-chip the
>> profiling data ourselves, and although the data looks okay, ‘llvm-profdata’
>> reports the following error:
>>
>>
>>

>> *llvm-profdata show -all-functions -counts -detailed-summary -text
>> -static-func-full-module-prefix ~/Downloads/default.profraw*

>>
>> error: ~/Downloads/default.profraw: Failed to uncompress data (zlib)
>>
>>
>>
>> But even for a hosted system, it is not clear to me where ‘zlib’ gets
>> involved in compressing the data. The start of our data looks like:
>>
>>
>>

>> *hexdump -C default.profraw*

[Xinliang David Li via llvm-dev]

I think that is a good idea.

David

[Martin J. O'Riordan via llvm-dev]

Thanks for the feedback.  I am off for a short well deserved vacation J, but when I get back I will put these suggestions to good use.

 

All the best,

 

            MartinO

[Martin J. O'Riordan via llvm-dev]

How can I build the profile reader with ZLIB support enabled? I configure and build all the tools together with 'clang', so I would have expected that all or none support ZLIB, or is this a cross-compiler configuration specific issue?

Thanks,

MartinO

-----Original Message-----
From: llvm-dev [mailto:llvm-dev...@lists.llvm.org] On Behalf Of Justin Bogner via llvm-dev
Sent: 18 July 2017 19:37
To: Xinliang David Li via llvm-dev <llvm...@lists.llvm.org>
Subject: Re: [llvm-dev] PGO, zlib and 'default.profraw'

[Xinliang David Li via llvm-dev]

set LLVM_ENABLE_ZLIB=ON with cmake invocation.  zlib should be installed and zlib.h header file needs to be in the header search path.

Is your llvm-profdata tool built together with clang? 

David

[ORiordan, Martin via llvm-dev]

Thanks.  Yes, I build ‘clang’ and all the other tools and utilities together, so I am surprised that ‘clang’ would have ‘zlib’ enabled but not ‘llvm-profdata’.  I will get back to this when I return on Monday, and I have a suspicion I may be doing something else wrong that I will formulate and ask next week.

 

Thanks for your help,

 

            MartinO

 

From: llvm-dev [mailto:llvm-dev...@lists.llvm.org] On Behalf Of Xinliang David Li via llvm-dev
Sent: Tuesday, July 18, 2017 10:41 PM
To: Martin J. O'Riordan <Mar...@theheart.ie>
Cc: LLVM Developers <llvm...@lists.llvm.org>
Subject: Re: [llvm-dev] PGO, zlib and 'default.profraw'

 

set LLVM_ENABLE_ZLIB=ON with cmake invocation.  zlib should be installed and zlib.h header file needs to be in the header search path.

[Martin J. O'Riordan via llvm-dev]

Hi David,

 

When I use CMake to configure, ‘zlib’ and its header are detected - I build on CentOS 6.5 or CentOS 7.  Since I run CMake from the command-line, I tried added ‘-DLLVM_ENABLE_ZLIB=0’ and ‘-DLLVM_ENABLE_ZLIB=1’ (using ‘-DLLVM_ENABLE_ZLIB=ON’ does not seem to work).  Both ‘clang’ and ‘llvm-profdata’ (and all other tools and utilities) are configured and built together, in any event, they are both built with ‘zlib’ enabled or ‘zlib’ disabled.

 

On Windows with VS2015 there is no ‘zlib’.

 

But I think that there is something different causing the problem, possibly in the linking.  When profiling instrumentation is enabled, there are 4 new sections introduced:

__llvm_prf_cnts

__llvm_prf_data

__llvm_prf_names

__llvm_prf_vnds

 

Since our target is an heterogeneous multi-core embedded system, it requires the use of some elaborate linking techniques, and the LD scripts are carefully crafted for this purpose.  We use ‘ld’ from Binutils v2.28.  However, I do not know what these new sections mean to ‘llvm-profdata’ or how they should be laid out.  I cannot find any reference to these sections in any of the in-tree targets for Binutils or for LLVM-LLD, so there is no guidance or examples on how they should be collated and ordered, or how they should be placed in the memory-map.

 

What I did is added a very simple line to my existing LD script’s rule for collating ‘.data’ as:

SECTIONS

{

  ...

  .data:

    {

      ...

      *(__llvm_prf_*)

      ...

    }

  ...

}

I’m guessing that this is too naïve and simplistic, and that something else is required, but I can’t find any sample LD scripts in either the LLD (v5.0) or Binutils (v2.28) sources.  I was considering replacing the line:

*(__llvm_prf_*)

with:

*(__llvm_prf_cnts)

*(__llvm_prf_data)

*(__llvm_prf_names)

*(__llvm_prf_vnds)

but without knowing what is required, this is still just a wild guess.

 

Any thoughts or recommendations on how these sections should be linked, and whether they need any special “PROVIDE” declarations?  Or “SORT” clauses?

 

Thanks,

 

            MartinO

[Xinliang David Li via llvm-dev]

The order of those sections in memory does not matter. What needed is a way to tell the profile runtime where to find the start and end of the sections. On Linux, the linker defines symbols __start_<section_name> and __stop_<section_name>. If your linker does not do that, you can use linker script to provide the start and end:

  __start___llvm_prf_data = ADDR(__llvm_prf_data);

  __start___llvm_prf_data = ADDR(__llvm_prf_data) + SIZEOF(__llvm_prf_data);

...

David

[Martin J. O'Riordan via llvm-dev]

Perfect, thanks David, this is exactly the kind of information I need.

 

Is this stated anywhere in the documentation?  I have not been able to find such an explanation.  I am also not getting unresolved symbols for ‘__start___llvm_prf_<name>’ or ‘__stop___llvm_prf_<name>’ - does the library use “weak” references to these?  How does ‘llvm-profdata’ locate the ELF file for the program when it only has ‘default.profraw’ as input? on an embedded system with no file-system “paths” don’t make a lot of sense.

 

My guess is that the profile library is not really used much by embedded targets, nor indeed the other instrumentation libraries.

 

Thanks again for your help and I will define these symbols during linking and let you know how things pan out.

 

            MartinO

 

From: Xinliang David Li [mailto:xinli...@gmail.com]
Sent: 25 July 2017 17:32
To: Martin J. O'Riordan <Mar...@theheart.ie>
Cc: llvm-dev <llvm...@lists.llvm.org>
Subject: Re: [llvm-dev] PGO, zlib and 'default.profraw'

 

The order of those sections in memory does not matter. What needed is a way to tell the profile runtime where to find the start and end of the sections. On Linux, the linker defines symbols __start_<section_name> and __stop_<section_name>. If your linker does not do that, you can use linker script to provide the start and end:

[Xinliang David Li via llvm-dev]

On Tue, Jul 25, 2017 at 10:07 AM, Martin J. O'Riordan <Mar...@theheart.ie> wrote:

Perfect, thanks David, this is exactly the kind of information I need.

 

Is this stated anywhere in the documentation?  I have not been able to find such an explanation. 

No, the implementation details like this is not documented.  The best source of information is the the source code in profiler runtime in projects/compiler-rt/lib/profile directory.

The start/stop symbol mechanism is supported for Linux, FreeBSD and Darwin.  Other platforms rely runtime registration of section start and end on a per module basis. This also requires the instrumentation pass to emit the registration code.

 

I am also not getting unresolved symbols for ‘__start___llvm_prf_<name>’ or ‘__stop___llvm_prf_<name>’ - does the library use “weak” references to these? 

The references are from one platform specific file in profile runtime. If that file is not linked in, the generic implementation (which relies on runtime registration) will be linked in (InstrProfilingPlatformOther.c). See CMakeLists.txt for details.   In your case, what likely happens is that the registration code is not emitted so the profiler runtime dumps empty data.

 

How does ‘llvm-profdata’ locate the ELF file for the program when it only has ‘default.profraw’ as input?

llvm-profdata does not need to locate ELF file.  llvm-cov tool, on the hand, requires it to locate the covmap section in the executable file.

David