[llvm-dev] Debug information with clang-cl on Windows

619 views
Skip to first unread message

Tobias Hieta via llvm-dev

unread,
Mar 9, 2021, 3:37:48 AM3/9/21
to llvm-dev
Hello,

We recently switched our windows builds from the CL compiler to
clang-cl. This was a pretty big jump in terms of code generation and
having a uniform compiler across all the major platforms.

Our setup is that our Windows developer use MSVC as the IDE and
compile with our own build of clang-cl and link with lld-link.

Some of the developers noticed that some of the debug information are
not as good as with CL. Or rather some variables seems to be optimized
out, this is what the IDE says, see attached screenshot. I have tried
to figure out if there is a rhyme and reason to what variables are
gone - but I have not been able to see any pattern.

Our command line is:

clang-cl.exe /nologo -DUNICODE -DUSE_BREAKPAD -DWIN32
-DWIN32_LEAN_AND_MEAN -DWINVER=0x0601 -D_CRT_NONSTDC_NO_DEPRECATE
-D_CRT_SECURE_NO_WARNINGS -D_LEGACY_NAT_TRAVERSAL_ -D_REENTRANT
-D_SCL_SECURE_NO_WARNINGS -D_SILENCE_ALL_CXX17_DEPRECATION_WARNINGS
-D_UNICODE -D_WIN32_WINNT=0x0601 -D_WINSOCK_DEPRECATED_NO_WARNINGS
-Iincludes -m32 -fdiagnostics-absolute-paths
--target=i686-windows-msvc -m32 /DWIN32 /D_WINDOWS /GR /EHsc /Zi /Ob0
/Od /RTC1 -MDd /Zm512 /EHsc /bigobj /W3 -Werror -Wsign-compare
-Wno-reorder-ctor -Wno-delete-non-virtual-dtor -Wunused-variable
-Wuninitialized -std:c++14

Is there any good way to debug this or is it a known limitation with
clang-cl or 32 bit (we don't do 64 yet - but with clang-cl we hope to
move towards that soon). Or am I missing some flags?

Thanks,
Tobias
108345285-bf402180-71e6-11eb-9090-29f6ebb8423e.png

Tobias Hieta via llvm-dev

unread,
Mar 9, 2021, 3:54:38 AM3/9/21
to llvm-dev
I forgot a pretty important piece of information, we are currently on
LLVM 11.0.1.

_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

David Blaikie via llvm-dev

unread,
Mar 9, 2021, 5:46:35 PM3/9/21
to Tobias Hieta, Reid Kleckner, llvm-dev
I would guess there's no one bug here - debug info quality (specifically/especially the location of variables when compiling with optimizations enabled) is a long tail/sliding scale of issues. LLVM is, on the whole, not spectacular at this - a matter of bugs to fix, some small/simple, others more systemic/representational.

If you're interested in contributing to this work - generally the first step is to isolate the problem - first thing you can do is if you have some experience dumping/examining debug info in object files, you can dump/examine the relevant object files then try to reduce/remove code from the relevant objects, etc, to see what the minimal example is that'll make it easier to figure out how to fix.

Adrian McCarthy via llvm-dev

unread,
Mar 9, 2021, 6:45:14 PM3/9/21
to David Blaikie, llvm-dev
I believe that, even with `/Od` (that's Microsoft for "disable optimizations"), clang-cl applies some optimizations.

People on my team have been fixing debug info quality bugs for Windows for a while.  I don't have specific knowledge of these issues, but it's possible newer versions (like 12 or head) may have solved this.  We're somewhat more focused on 64-bit than 32-bit builds, so you might encounter some differences there.

If you can reproduce the problem with a reduced code sample, please file a bug with all the deets.  If you do some debugging yourself, please let us know what you find.

Tobias Hieta via llvm-dev

unread,
Mar 10, 2021, 2:26:13 PM3/10/21
to Adrian McCarthy, llvm-dev
Hello,

Thanks for your reply!

Is it possible to disable optimisations with clang-cl and see if that makes a difference?

I will also try to build with clang 12 instead and see if there is any improvement. 

Happy to try to reproduce or fix the issue - but I am unfamiliar with this debug info on windows and how to even start looking into it, any pointers are appreciated. 

Thanks,
Tobias. 

Alexandre Ganea via llvm-dev

unread,
Mar 10, 2021, 3:02:14 PM3/10/21
to Tobias Hieta, Adrian McCarthy, James Henderson, llvm...@lists.llvm.org

 

 

De : Alexandre Ganea
Envoyé : March 10, 2021 3:02 PM
À : 'Tobias Hieta' <tob...@plexapp.com>; Adrian McCarthy <amcc...@google.com>; 'James Henderson' <jh737...@my.bristol.ac.uk>
Objet : RE: [llvm-dev] Debug information with clang-cl on Windows

 

DExTer can probably help bisecting to the specific optimization pass(es) that cause the debug infos to be omitted? https://lists.llvm.org/pipermail/llvm-dev/2018-August/125780.html

 

+James.

 

De : llvm-dev <llvm-dev...@lists.llvm.org> De la part de Tobias Hieta via llvm-dev
Envoyé : March 10, 2021 2:26 PM
À : Adrian McCarthy <amcc...@google.com>
Cc : llvm-dev <llvm...@lists.llvm.org>
Objet : Re: [llvm-dev] Debug information with clang-cl on Windows

Reid Kleckner via llvm-dev

unread,
Mar 10, 2021, 3:38:27 PM3/10/21
to Tobias Hieta, llvm-dev
I'm glad to hear things are working reasonably well for you. :)

Regarding variables marked as optimized out, the main issue that I'm aware of has to do with parameters at function entry. We had a user report this as https://crbug.com/755841.

What you have in the screenshot doesn't seem related to parameters. It seems like this issue could be specific to 32-bit x86. We can definitely take a look at a more detailed bug report if you find the time to file one, but as Adrian mentioned, 64-bit is becoming the better tested, more robust, target.

Reid

On Tue, Mar 9, 2021 at 12:37 AM Tobias Hieta via llvm-dev <llvm...@lists.llvm.org> wrote:

Greg Bedwell via llvm-dev

unread,
Mar 10, 2021, 3:48:33 PM3/10/21
to Alexandre Ganea, llvm...@lists.llvm.org
llvm-dva (not yet in tree but see 
https://reviews.llvm.org/D88661 ) might also be useful here depending on how mature the codeview support is (cc Carlos - please correct me if anything I said is wrong!). 

I think, in theory it should allow us to do a direct comparison between the debug info generated by clang-cl, and that generated by cl.exe for the same code (and also with clang's DWARF output if that's possible to generate too) so we can see exactly what the differences are and what's missing.

When I previously looked into some of this stuff using dexter I found a few examples like 
https://bugs.llvm.org/show_bug.cgi?id=37682 where clang was comparing unfavorably with MSVC because it was doing more aggressive loop optimizations. I don't know how widespread it is or whether it's still the case in more recent versions, but I certainly ran into it on more than one occasion where MSVC was doing things like leaving stores in loops whereas clang would hoist it instead for better codegen but an unfortunately worse debugging experience. 

-Greg



via llvm-dev

unread,
Mar 11, 2021, 2:57:33 AM3/11/21
to gregb...@gmail.com, alexand...@ubisoft.com, llvm...@lists.llvm.org

Hi Greg,

 

You are correct. I am including the section ‘Comparing toolchains’ from the llvm-dva

documentation, for a specific debug information issue detected in Clang.

 

See: https://bugs.llvm.org/show_bug.cgi?id=44884

 

-Carlos

 

Comparing toolchains

--------------------

 

In the previous section, we compared GCC and Clang. The current implementation

of llvm-dva have sufficient support for CodeView format, making possible the

comparison between MSVC and Clang compilers.

 

-----------------------------------------------------------------------

pr_44884.cpp

-----------------------------------------------------------------------

1  int bar(float Input) { return (int)Input; }

 3  unsigned foo(char Param) {

4    typedef int INT;                      // ** Definition for INT **

5    INT Value = Param;

6    {

7      typedef float FLOAT;                // ** Definition for FLOAT **

8      {

9        FLOAT Added = Value + Param;

10        Value = bar(Added);

11      }

12    }

13    return Value + Param;

14  }

 

The above test (from PR44884) is used to illustrates a scope issue found in

the Clang compiler.

 

See: https://bugs.llvm.org/show_bug.cgi?id=44884

 

The lines 4 and 7 contains 2 typedefs, defined at different lexical scopes.

4    typedef int INT;

7      typedef float FLOAT;

 

These are the logical views that llvm-dva generates for 3 different compilers

(MSVC, Clang and GCC), emitting different debug info formats (CodeView, DWARF)

on different platforms.

 

-----------------------------------------------------------------------

pr_44884_dw.o - Compiled with Clang (DWARF format).

-----------------------------------------------------------------------

Logical View:

[000]           {File} 'pr_44884_dw.o' -> elf64-x86-64

[001]             {CompileUnit} 'pr_44884.cpp'

[002]               {Producer} 'clang version 11.0.0

[002]     7         {Function} extern not_inlined 'bar' -> 'int'

[003]     7           {Parameter} 'Input' -> 'float'

[002]     9         {Function} extern not_inlined 'foo' -> 'unsigned int'

[003]                 {Block}

[004]    15             {Variable} 'Added' -> 'FLOAT'

[003]     9           {Parameter} 'Param' -> 'char'

[003]    13           {TypeAlias} 'FLOAT' -> 'float'

[003]    10           {TypeAlias} 'INT' -> 'int'

[003]    11           {Variable} 'Value' -> 'INT'

 

-----------------------------------------------------------------------

pr_44884_gc.o - Compiled with GCC (DWARF Format).

-----------------------------------------------------------------------

Logical View:

[000]           {File} 'pr_44884_gc.o' -> elf64-x86-64

[001]             {CompileUnit} 'pr_44884.cpp'

[002]               {Producer} 'GNU C++ 5.5.0 20171010'

[002]     7         {Function} extern not_inlined 'bar' -> 'int'

[003]     7           {Parameter} 'Input' -> 'float'

[002]     9         {Function} extern not_inlined 'foo' -> 'unsigned int'

[003]                 {Block}

[004]                   {Block}

[005]    15               {Variable} 'Added' -> 'FLOAT'

[004]    13             {TypeAlias} 'FLOAT' -> 'float'

[003]     9           {Parameter} 'Param' -> 'char'

[003]    10           {TypeAlias} 'INT' -> 'int'

[003]    11           {Variable} 'Value' -> 'INT'

 

-----------------------------------------------------------------------

pr_44884_cv.o - Compiled with Clang (CodeView format).

-----------------------------------------------------------------------

Logical View:

[000]           {File} 'pr_44884_cv.o' -> COFF-x86-64

[001]             {CompileUnit} 'pr_44884.cpp'

[002]               {Producer} 'clang version 11.0.0

[002]               {Function} extern not_inlined 'bar' -> 'int'

[003]                 {Parameter} 'Input' -> 'float'

[002]               {Function} extern not_inlined 'foo' -> 'unsigned'

[003]                 {Block}

[004]                   {Variable} 'Added' -> 'float'

[003]                 {Parameter} 'Param' -> 'char'

[003]                 {TypeAlias} 'FLOAT' -> 'float'

[003]                 {TypeAlias} 'INT' -> 'int'

[003]                 {Variable} 'Value' -> 'int'

 

-----------------------------------------------------------------------

pr_44884_ms.o - Compiled with MSVC (CodeView Format).

-----------------------------------------------------------------------

Logical View:

[000]           {File} 'pr_44884_ms.o' -> COFF-i386

[001]             {CompileUnit} 'pr_44884.cpp'

[002]               {Producer} 'Microsoft (R) Optimizing Compiler'

[002]               {Function} extern not_inlined 'bar' -> 'int'

[003]                 {Parameter} 'Input' -> 'float'

[002]               {Function} extern not_inlined 'foo' -> 'unsigned'

[003]                 {Block}

[004]                   {Block}

[005]                     {Variable} 'Added' -> 'float'

[004]                   {TypeAlias} 'FLOAT' -> 'float'

[003]                 {Parameter} 'Param' -> 'char'

[003]                 {TypeAlias} 'INT' -> 'int'

[003]                 {Variable} 'Value' -> 'int'

 

From the previous logical views, we can see that the Clang compiler emits both

typedefs at the same lexical scope (3), which is wrong, while GCC and MSVC emit

correct lexical scope for both typedefs.

 

---------+----------+----------------------------------------------------------

Compiler | Format   | Lexical Scope

---------|----------|----------------------------------------------------------

Clang    | DWARF    | [003]    13           {TypeAlias} 'FLOAT' -> 'float'

         |          | [003]    10           {TypeAlias} 'INT' -> 'int'

---------|----------+----------------------------------------------------------

GCC      | DWARF    | [004]    13             {TypeAlias} 'FLOAT' -> 'float'

         |          | [003]    10           {TypeAlias} 'INT' -> 'int'

---------|----------|----------------------------------------------------------

Clang    | CodeView | [003]                 {TypeAlias} 'FLOAT' -> 'float'

         |          | [003]                 {TypeAlias} 'INT' -> 'int'

---------|----------|----------------------------------------------------------

MSVC     | CodeView | [004]                   {TypeAlias} 'FLOAT' -> 'float'

         |          | [003]                 {TypeAlias} 'INT' -> 'int'

---------+----------+----------------------------------------------------------

 

Note: One of the main limitations while processing CodeView debug info, is the

reduced line information emitted for types and symbols, making difficult to use

the comparison feature within llvm-dva, as the line numbers are one of the

criteria for logical element match. In the meantime, any graphical comparison

tool is able to compare and show the logical view differences.

 

The above table shows the omitted line numbers for the referenced typedefs.




**********************************************************************
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify
siee.po...@sony.com
This footnote also confirms that this email message has been checked for all known viruses.
Sony Interactive Entertainment Europe Limited
Registered Office: 10 Great Marlborough Street, London W1F 7LP, United Kingdom
Registered in England: 3277793
**********************************************************************

P
Please consider the environment before printing this e-mail

James Henderson via llvm-dev

unread,
Mar 11, 2021, 3:57:40 AM3/11/21
to Alexandre Ganea, jeremy...@sony.com, llvm...@lists.llvm.org
I'm not the person to ask DExTer questions (I just fixed a few tests that were causing me bother with regards to the cross-project-tests proposal). You'll want someone like Jeremy Morse (CC'ed).

James

Jeremy Morse via llvm-dev

unread,
Mar 11, 2021, 6:19:36 AM3/11/21
to jh737...@my.bristol.ac.uk, llvm...@lists.llvm.org, jeremy...@sony.com
Hi,

On the Dexter front, I've found it most useful for small tests in a
very tight loop, i.e. modifying a source file and quickly getting some
feedback about what kind of debug-info defects are present.
Alternately, with opt-bisect-limit to find passes that contribute
defects. However, it doesn't work so well with large code bases, we're
actively pursing improving that.

There are a few variable-location-coverage improvements in the works
-- Stephens patches [0] for variadic variable locations are just
landing, although I don't believe they'll be useful for CodeView/PDB
targets.

More bug reports about variable locations dropped by clang would be
great as we receive very few of them, they're usually not a blocker to
developers. Any patterns in what variables go missing would be
especially interesting, i.e. "they're usually in lambdas" or "many are
loop iterators". That'd help us focus increased testing on those
areas.

[0] https://reviews.llvm.org/D91722

--
Thanks,
Jeremy

Reply all
Reply to author
Forward
0 new messages