decoding DWARF data

[stephen.t....@gmail.com]

(This is possibly off-topic)

I'm trying to decode the DWARF data in an ELF binary and I'm not sure how to handle functions.

I'm using the elf/dwarf package in the Go standard library and successfully using it to identify compile units and source files and also to find line entries for an executed address.

What I can't see is how to relate a dwarf.LineEntry to a function (or SubProgram as it is called in DWARF terminology).

If the SubProgram TAG has both Low PC and High PC attributes then I can see how it can be done. Where it gets tricky is if a SubProgram has been inlined.

What I don't fully understand is how InlinedSubPrograms relate to SubPrograms.

I realise that this isn't really a Go question but I'm hoping someone has used the Go standard library in this way and understands how to solve this problem.

Thanks.

[Ian Lance Taylor]

The DWARF line number information maps PC values to a file name and a
line number. It sounds like you want to map a PC value to a function,
and you want to consider inlining information. To do that use
https://pkg.go.dev/debug/dwarf#Reader.SeekPC to get an Entry and look
for the AttrName or AttrLinkageName attribute. Note that, as the
documentation says, SeekPC is not efficient if you need to look up a
bunch of PC values. For that you'll want to loop through the entries,
call the Ranges method, and build a lookup table.

Hope this helps.

Ian

[stephen.t....@gmail.com]

Reader.SeekPC() is useful. Thanks. The problem was due to my misunderstanding of how inlined functions are represented in the DWARF data.

I've solved it now. Thanks again.

[Than McIntosh]

For general information on DWARF and inlining, you might look at this design doc:

https://github.com/golang/proposal/blob/master/design/22080-dwarf-inlining.md#example

which describes the changes made to the compiler to add DWARF inlining support. HTH.

Thanks, Than

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/6b1b4e4f-e9af-4954-9296-de50b4d548f5n%40googlegroups.com.

[stephen.t....@gmail.com]

That does help.

I was confused because I was seeing InlinedSubroutines linking back to an abstract Subroutine, but without a corresponding concrete Subroutine. But according to the document, if all calls to the abstract Subroutine are inlined the concrete Subroutine is not included in the DWARF data, leaving only the abstract Subroutine and the InlinedSubroutines. That makes sense.

Thanks, Steve

[stephen.t....@gmail.com]

I've encountered something else that I don't understand. This time regarding dwarf.AttrDeclFile and how the value of that field relates to the list of source files in the project

From what I can understand, the value of AttrDeclFile is an index into an array of filenames. If the number is zero then the indicates that no source file has been specified.

The dwarf package allows retrieval of the file list with LineReader.Files() function - the LineReader being created with reference to the current compilation unit.

For most ELF/DWARF examples I have, my understanding seems correct. However, in one example I have, the value of AttrDeclFile can exceed the length of the file array returned by LineReader.Files(). An invalid reference in other words.

Moreover, for AttrDeclFiles values that are within range of the array, they are most definitely referencing the wrong file.

I'm I misunderstanding the relationship between AttrDeclFile and LineReader.Files(), is there some other way of accessing the file array, or has something gone horribly wrong?

Regards, Steve.

[Ian Lance Taylor]

On Sun, Feb 13, 2022 at 1:43 PM stephen.t....@gmail.com
<stephen.t....@gmail.com> wrote:
>
> I've encountered something else that I don't understand. This time regarding dwarf.AttrDeclFile and how the value of that field relates to the list of source files in the project
>
> From what I can understand, the value of AttrDeclFile is an index into an array of filenames. If the number is zero then the indicates that no source file has been specified.
>
> The dwarf package allows retrieval of the file list with LineReader.Files() function - the LineReader being created with reference to the current compilation unit.
>
> For most ELF/DWARF examples I have, my understanding seems correct. However, in one example I have, the value of AttrDeclFile can exceed the length of the file array returned by LineReader.Files(). An invalid reference in other words.
>
> Moreover, for AttrDeclFiles values that are within range of the array, they are most definitely referencing the wrong file.
>
> I'm I misunderstanding the relationship between AttrDeclFile and LineReader.Files(), is there some other way of accessing the file array, or has something gone horribly wrong?

What you write sounds correct. Can you show us the output "readelf
--debug" on the file in question, and point out the problematic case?
Thanks.

Ian

[Stephen Illingworth]

Okay. I've attached a text file showing the results of "readelf --debug". The ELF file itself was produced by GCC 10.3.1, part of the toolchain distributed by ARM.

These are the dwarf.Entry instances as output from my Go program, that are out of range for the result of LineReader.Files()

&dwarf.Entry{Offset:0x37a1, Tag:dwarf.TagSubprogram, Children:true, Field:[]dwarf.Field{dwarf.Field{Attr:dwarf.AttrExternal, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrName, Val:"clearBuffer", Class:dwarf.ClassString}, dwarf.Field{Attr:dwarf.AttrDeclFile, Val:10, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclLine, Val:426, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclColumn, Val:6, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrPrototyped, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrSibling, Val:0x37c9, Class:dwarf.ClassReference}}}

&dwarf.Entry{Offset:0x3767, Tag:dwarf.TagSubprogram, Children:true, Field:[]dwarf.Field{dwarf.Field{Attr:dwarf.AttrExternal, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrName, Val:"ColourConvert", Class:dwarf.ClassString}, dwarf.Field{Attr:dwarf.AttrDeclFile, Val:10, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclLine, Val:462, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclColumn, Val:15, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrPrototyped, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrType, Val:0x230f, Class:dwarf.ClassReference}, dwarf.Field{Attr:dwarf.AttrSibling, Val:0x3786, Class:dwarf.ClassReference}}}

&dwarf.Entry{Offset:0x3228, Tag:dwarf.TagSubprogram, Children:false, Field:[]dwarf.Field{dwarf.Field{Attr:dwarf.AttrExternal, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrName, Val:"processAnimationCommand", Class:dwarf.ClassString}, dwarf.Field{Attr:dwarf.AttrDeclFile, Val:10, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclLine, Val:1492, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclColumn, Val:6, Class:dwarf.ClassConstant}}}

&dwarf.Entry{Offset:0x3690, Tag:dwarf.TagSubprogram, Children:true, Field:[]dwarf.Field{dwarf.Field{Attr:dwarf.AttrExternal, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrName, Val:"InitGameX", Class:dwarf.ClassString}, dwarf.Field{Attr:dwarf.AttrDeclFile, Val:10, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclLine, Val:567, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclColumn, Val:6, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrSibling, Val:0x36f4, Class:dwarf.ClassReference}}}

&dwarf.Entry{Offset:0x3786, Tag:dwarf.TagSubprogram, Children:true, Field:[]dwarf.Field{dwarf.Field{Attr:dwarf.AttrExternal, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrName, Val:"setVideoBufferPointers", Class:dwarf.ClassString}, dwarf.Field{Attr:dwarf.AttrDeclFile, Val:10, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclLine, Val:437, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclColumn, Val:6, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrSibling, Val:0x37a1, Class:dwarf.ClassReference}}}

&dwarf.Entry{Offset:0x3204, Tag:dwarf.TagSubprogram, Children:true, Field:[]dwarf.Field{dwarf.Field{Attr:dwarf.AttrExternal, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrName, Val:"setAnimation", Class:dwarf.ClassString}, dwarf.Field{Attr:dwarf.AttrDeclFile, Val:10, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclLine, Val:1544, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclColumn, Val:6, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrPrototyped, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrSibling, Val:0x321f, Class:dwarf.ClassReference}}}

&dwarf.Entry{Offset:0x3664, Tag:dwarf.TagSubprogram, Children:true, Field:[]dwarf.Field{dwarf.Field{Attr:dwarf.AttrExternal, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrName, Val:"Initialize", Class:dwarf.ClassString}, dwarf.Field{Attr:dwarf.AttrDeclFile, Val:10, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclLine, Val:688, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclColumn, Val:6, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrSibling, Val:0x3690, Class:dwarf.ClassReference}}}

&dwarf.Entry{Offset:0x375a, Tag:dwarf.TagSubprogram, Children:false, Field:[]dwarf.Field{dwarf.Field{Attr:dwarf.AttrExternal, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrName, Val:"main", Class:dwarf.ClassString}, dwarf.Field{Attr:dwarf.AttrDeclFile, Val:10, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclLine, Val:490, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclColumn, Val:5, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrType, Val:0x2327, Class:dwarf.ClassReference}}}

&dwarf.Entry{Offset:0x35f8, Tag:dwarf.TagSubprogram, Children:false, Field:[]dwarf.Field{dwarf.Field{Attr:dwarf.AttrExternal, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrName, Val:"InitGameDatastreams", Class:dwarf.ClassString}, dwarf.Field{Attr:dwarf.AttrDeclFile, Val:10, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclLine, Val:797, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclColumn, Val:6, Class:dwarf.ClassConstant}}}

&dwarf.Entry{Offset:0x34f4, Tag:dwarf.TagSubprogram, Children:true, Field:[]dwarf.Field{dwarf.Field{Attr:dwarf.AttrExternal, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrName, Val:"GameScheduleAnimate", Class:dwarf.ClassString}, dwarf.Field{Attr:dwarf.AttrDeclFile, Val:10, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclLine, Val:1077, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclColumn, Val:6, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrSibling, Val:0x352d, Class:dwarf.ClassReference}}}

&dwarf.Entry{Offset:0x3601, Tag:dwarf.TagSubprogram, Children:true, Field:[]dwarf.Field{dwarf.Field{Attr:dwarf.AttrExternal, Val:true, Class:dwarf.ClassFlag}, dwarf.Field{Attr:dwarf.AttrName, Val:"Scheduler", Class:dwarf.ClassString}, dwarf.Field{Attr:dwarf.AttrDeclFile, Val:10, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclLine, Val:753, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrDeclColumn, Val:6, Class:dwarf.ClassConstant}, dwarf.Field{Attr:dwarf.AttrSibling, Val:0x3629, Class:dwarf.ClassReference}}}

[stephen.t....@gmail.com]

I've figured out what the problem is. The problem in my code was caused by the fact that this ELF/DWARF file has so many compile units, whereas my other examples have far fewer compile units. I'll try to explain what my mistake was:

Task: Looking for the function declaration file/line, for an executable address

Original Solution (pseudocode):

new DWARF Reader

for every ENTRY

    if ENTRY is compile unit

        initialise new LINEREADER

    if ENTRY is SUBPROGRAM or INLINEDSUBROUTINE

        if ADDRESS is in range of ENTRY

            if has ABSTRACT ORIGIN tag

                seek ABSTRACT offset      <--- this is where the bug is        

       

       look up DECL FILE and DECL LINE using LINEREADER

The issue was caused when following the abstract origin. In this ELF/DWARF file, it is likely that the entry is in a different compile unit, meaning that the line reader is referring to the wrong line table.

With a bit more care, I'm now using the correct line reader and the problem no longer occurs.

Thanks for the help.

[stephen.t....@gmail.com]

Hello,

Further to my previous questions on the subject of DWARF: I'm having difficulty
understanding how to handle relocatable DWARF data using the standard library.

Currently, I am successfully loading an ELF file and relocating it in memory
(the data is being loaded into an ARM emulator and runs correctly).

However, I'm puzzled at how the elf.DWARF() function in standard library
handles relocation. Looking at the source for this function I can see that it
will process any REL or RELA sections related to the debug sections. This is a
big help but I don't understand how it takes into account the already relocated
ELF sections.

For instance, when I'm processing the line entries for a compile unit
(retrieved via a dwarf.LineReader) the Address field of each LineEntry instance
is the unrelocated PC value for that instruction. Currently, I'm handling this
by adjusting the Address field by the origin address of the relocated .text
section, as I encounter them. This works but it feels like I'm missing
something.

Do I need to adjust the address field manually or is there a step I'm
misunderstanding?

Stephen