Debug format for Windows
Remi Gillig
I was reading this page [1], and was wondering what was the status quo of debug information in Windows binaries. I am planning on doing a Visual Studio 2010 support for Go and I would love to be able to debug inside Visual Studio (by creating a custom debug engine, see [2]) but if the compiler does not generate any debug information in Windows binaries I would be disappointed. Are there any plans to write any format? I don't think the PDB format can handle anything else than C++.
Remi Gillig.
[1] http://blog.golang.org/2010/11/debugging-go-code-status-report.html
[2] http://msdn.microsoft.com/en-us/library/bb145934.aspx
Mateusz Czapliński
> I was reading this page [1], and was wondering what was the status quo of
> debug information in Windows binaries.
think it is working for gdb on Windows.
I've been working close to this area about a month ago, and from
resulting discussion it seems the suggested long(er) term direction is
to emit all symbols into a PE/COFF symbols table (see [1] and further
comments).
[1] http://codereview.appspot.com/4894051/#msg12
With regards to PDB, I believe I've read some comment stating that
it's an undocumented format, but I'm totally not sure about that.
regards
/Mateusz Czaplinski.
Remi Gillig
Visual Studio. However, it seems the COFF format does not contain line
numbers [1]. Or maybe this reads as "Visual Studio does not put line
numbers in COFF format". I did not read the specification for COFF
yet.
I agree that PDB should not be the way to go. It seems the CodeView
format is more documented but the specification is hard to find.
Anyway, I don't want to implement a format known by Visual Studio, I
will be glad with anything. It was just that the blog post was stating
there was no debug information in Windows binaries and I wanted an
update on that.
I will study all that and get back to you if have any questions.
Thanks !
Remi Gillig.
[1] http://msdn.microsoft.com/en-us/library/aa260783%28v=vs.60%29.aspx
Ian Lance Taylor
> I read some docs about COFF and CodeView formats, both supported by
> Visual Studio. However, it seems the COFF format does not contain line
> numbers [1]. Or maybe this reads as "Visual Studio does not put line
> numbers in COFF format". I did not read the specification for COFF
> yet.
For the record, COFF does support line numbers natively. For many
targets it uses a 16-bit field for the line number, which can be
limiting. More seriously, COFF only stores a line number; it has no way
of associating the line number with a specific file. This makes it
nearly useless for C++, or for Go. I don't know what Visual Studio
does.
Ian
Remi Gillig
features supported by a debug format on Windows are on par with what
is possible with DWARF on other platforms. I don't know if it would
even be possible to output DWARF even for Windows binaries.
It doesn't matter what Visual Studio does too, I don't think it could
be reused directly anyway because their debugger only supports C++ and
MSIL (.NET).
So as of today, you can have the symbols from a Windows executable
with the nm command? Am I right?
Remi Gillig.
On Oct 5, 7:44 pm, Ian Lance Taylor <i...@google.com> wrote:
Hector Chu
> I don't know if it would
> even be possible to output DWARF even for Windows binaries.
that the binaries contain DWARF information.
Krzysztof Kowalczyk
not just for Visual Studio. Any compiler, debugger, profiler, code
coverage tool, decompiler, code rewriting tool etc., from Microsoft or
anyone else, understands PDB format.
PDB certainly handles C,C++ and .NET (C#). It probably can represent more.
Basic, read-only support is baked directly into windows in the form of
DbgHelp.dll (http://msdn.microsoft.com/en-us/library/windows/desktop/ms679294(v=vs.85).aspx)
and a richer interface via DIA SDK
(http://msdn.microsoft.com/en-us/library/t6tay6cz(v=VS.100).aspx)
The format is not officially documented. However, Microsoft released
an open-source, C# library for PDB format. Mono folks use it e.g. to
convert pdb files to mono's native mdb format:
https://github.com/mono/mono/tree/master/mcs/tools/pdb2mdb
https://github.com/mono/mono/blob/master/mcs/tools/pdb2mdb/CvInfo.cs
is a pretty good description of the format.
-- kjk
2011/10/5 Remi Gillig <remig...@gmail.com>:
Remi Gillig
for my purpose. I don't know how they handle both C++ and .NET in PDBs
though, this page contradicts this.
Thanks Hector, I didn't have the time to check everything yet, that
would help a lot indeed. Does that mean you can use gdb in Windows to
debug the binaries then?
Remi Gillig.
[1] http://msdn.microsoft.com/en-us/library/bb145316.aspx
On Oct 5, 11:01 pm, Krzysztof Kowalczyk <kkowalc...@gmail.com> wrote:
> PDB is THE format for providing symbol information on Windows. It's
> not just for Visual Studio. Any compiler, debugger, profiler, code
> coverage tool, decompiler, code rewriting tool etc., from Microsoft or
> anyone else, understands PDB format.
>
> PDB certainly handles C,C++ and .NET (C#). It probably can represent more.
>
> Basic, read-only support is baked directly into windows in the form of
> (http://msdn.microsoft.com/en-us/library/t6tay6cz(v=VS.100).aspx)
>
> The format is not officially documented. However, Microsoft released
> an open-source, C# library for PDB format. Mono folks use it e.g. to
> convert pdb files to mono's native mdb format:https://github.com/mono/mono/tree/master/mcs/tools/pdb2mdb
>
> https://github.com/mono/mono/blob/master/mcs/tools/pdb2mdb/CvInfo.cs
> is a pretty good description of the format.
>
> -- kjk
>
brainman
... Does that mean you can use gdb in Windows to
debug the binaries then?
Yes.
Alex
brainman
On Thursday, 6 October 2011 06:09:09 UTC+10, Remi Gillig wrote:
... I don't know if it would
even be possible to output DWARF even for Windows binaries.
DWARF information is stored in some sections of pe executable that 8l/6l generate.
So as of today, you can have the symbols from a Windows executable
with the nm command?
Native golang 6nm is supported since http://goo.gl/Dtpwp, but not others http://code.google.com/p/go/issues/detail?id=2199.
Alex
Krzysztof Kowalczyk
http://msdn.microsoft.com/en-us/library/bb145316.aspx implies in that
context.
I don't see which part of that page implies PDBs don't support C#.
From http://www.wintellect.com/CS/blogs/jrobbins/archive/2009/05/11/pdb-files-what-every-developer-must-know.aspx:
"A .NET PDB only contains two pieces of information, the source file
names and their lines and the local variable names. All the other
information is already in the .NET metadata so there is no need to
duplicate the same information in a PDB file."
Or: compile a C# program with Microsoft tools and check that a PDB
files is generated (if the right flags are used).
That page isn't really talking about PDBs but about extending Visual
Studio IDE with a custom debugging engine. They explicitly say that if
a tool generates PDB files, this is not necessary. It's only needed if
you want to implement a more esoteric scenario like e.g. remotely
debugging Linux executables running on Linux hosts from within Visual
Studio IDE (see e.g. how Mono folks have Visual Studio plugin that
allows debugging Mono with Visual Studio IDE
http://mono-tools.com/Debug.aspx).
You could use that Visual Studio extensibility to implement a Go
debugger hosted inside Visual Studio but it would be much simpler to
generate PDB files and it would work with any Windows tool, not just
Visual Studio.
-- kjk
2011/10/5 Remi Gillig <remig...@gmail.com>:
Joseph Poirier
> PDB is THE format for providing symbol information on Windows. It's
> not just for Visual Studio. Any compiler, debugger, profiler, code
> coverage tool, decompiler, code rewriting tool etc., from Microsoft or
> anyone else, understands PDB format.
I'd be interested in links to said compilers.
-joe
Joseph Poirier
> Krzysztof, from this page [1], I got the idea that PDB is not suited
> for my purpose. I don't know how they handle both C++ and .NET in PDBs
> though, this page contradicts this.
>
> Thanks Hector, I didn't have the time to check everything yet, that
> would help a lot indeed. Does that mean you can use gdb in Windows to
> debug the binaries then?
>
> Remi Gillig.
>
Possibly some useful/helpful information here:
http://dsource.org/projects/cv2pdb
http://tinyurl.com/ycu2gkc
-joe
j...@webmaster.ms
четверг, 6 октября 2011 г. 8:12:19 UTC+4 пользователь Joe Poirier написал:
Joseph Poirier
The Intel compiler is a front-end that needs to hook into a back-end,
e.g. into MS VC++ or GCC. Last I knew (version 11.0) the Intel
compiler (front-end) only generated DWARF2 and coff, but when it's
hooked into MS VC++ it can have VC++'s back-end linker generate a pdb
file. I could be wrong about VC++'s linker generating the pdb file
though.
There aren't that many C/C++ compilers for Windows (8 or so) and pdg
is exclusive to Microsoft, as far as I know.
Full Compilers:
TCC
Pelles C
Watcom C/C++
DIgital Mars C/C++
DeSmet C
LCC
Portland Group C/C++
... not many more
Compiler front-ends:
Intel C++
Comeau C/C++
Edison Design Group (they make C/C++ front-ends, think Intel, TI, MS)
-joe
John David Duncan
For me this brings up a really interesting question that (I admit) is
quite irrelevant to the go nuts. I'll make it quick: Microsoft
doesn't support much of C99. Do you know if I can use the Intel front
end plus the Microsoft backend to compile C99 code on Windows?
Thanks,
JD
Joseph Poirier
<john.dav...@gmail.com> wrote:
> Hi Joseph,
>
> For me this brings up a really interesting question that (I admit) is quite
> irrelevant to the go nuts. I'll make it quick: Microsoft doesn't support
> much of C99. Do you know if I can use the Intel front end plus the
> Microsoft backend to compile C99 code on Windows?
>
> Thanks,
> JD
I don't think Intel's front-end is fully C99 compliant. I heard a
rumor that the current version of Visual Studio had somewhere around
98% C99 compliance?
IMHO, one of the best C/C++ compliers for windows is from The Portland
Group. The C/C++ compiler generates some of the fastest code you'll
find, it comes with a nice Graphical debugger and a profiler (both
will do local MPI), and is C99 compliant. Also, the Comeau Computing
front-end for VC++ works nice and is 100% C99 compliant, although I
haven't used it for 4 or 5 years.
-joe
Paulo Pinto
development
don't care that much about C99 support.
Here is where they want to focus.
Personally I don't like their argument, specially when we see that C99
support
is so high in the Connect wish list. But then again, I do understand
the economic
argumentation.
--
Paulo
On Oct 19, 3:23 am, John David Duncan <john.david.dun...@gmail.com>
wrote:
Paulo Pinto
compilers I never
seen being used here in Europe.
Intel C++ is a full compiler with its own set of compiler backend
optimizations.
Where did you get this information from?
Joseph Poirier
> Their main argument is that their customers that do Windows only
> development
> don't care that much about C99 support.
>
> Here is where they want to focus.
I guess it makes sense to focus on C++ because most of the Windows
specific things like COM are in C++.
-joe
Joseph Poirier
> Funny that you did not mention C++ Builder, while mentioning several
> compilers I never
> seen being used here in Europe.
Isn't C++ builder just an IDE? Notable omissions from the list,
clang\LLVM and GCC
>
> Intel C++ is a full compiler with its own set of compiler backend
> optimizations.
>
You're right that it does do backend optimizations, but I wouldn't say
it's a full compiler because it requires parts of either GCC's or
Visual Studio's back-end. But it's definitely not just a front-end
either as I had _incorrectly_ stated.
-joe
Paulo Pinto
On Oct 19, 9:40 am, Joseph Poirier <jdpoir...@gmail.com> wrote:
> > Funny that you did not mention C++ Builder, while mentioning several
> > compilers I never
> > seen being used here in Europe.
>
> Isn't C++ builder just an IDE? Notable omissions from the list,
> clang\LLVM and GCC
originally
developed at Borland with Delphi and C++ Builder. And the product do
offer
interoperability between both languages.
The compiler is also available as command line tool (bcc).
>
>
>
> > Intel C++ is a full compiler with its own set of compiler backend
> > optimizations.
>
> You're right that it does do backend optimizations, but I wouldn't say
> it's a full compiler because it requires parts of either GCC's or
> Visual Studio's back-end. But it's definitely not just a front-end
> either as I had _incorrectly_ stated.
>
> -joe
Windows
most developers use IDEs and Visual Studio is what most Windows shops
use anyway.
But I can be wrong as I never used Intel C++ myself, hence I was
asking where you
got the information from.
--
Paulo
Aram Hăvărneanu
> it's a full compiler because it requires parts of either GCC's or
> Visual Studio's back-end. But it's definitely not just a front-end
> either as I had _incorrectly_ stated.
I doesn't require any back-end. It's a full compiler that you can run
from the command line if you don't have Visual Studio. It works. I
don't have Visual Studio.
--
Aram Hăvărneanu
Aram Hăvărneanu
> Windows
> most developers use IDEs and Visual Studio is what most Windows shops
> use anyway.
>
Actually it doesn't. You can run it without Visual Studio (and you
can run Microsoft compiler without Visual Studio as well, just install
WDK, the Windows Driver Kit, it's a tarball of the build tools that
use to build Windows with).
--
Aram Hăvărneanu
Paulo Pinto
Joseph Poirier
To get the Visual Studio command line driven compiler tools you need
to install Visual Studio. Not talking about the IDE. The last version
of the Intel compiler I used was for 32-bit platforms and it required
GCC when installed on OSX or Linux and Visual Studio's command line
compiler tools on Windows.
-joe
Aram Hăvărneanu
> to install Visual Studio.
No, you don't. Just get the WDK:
http://msdn.microsoft.com/en-us/windows/hardware/gg487463
It's a tarball of the build infrastructure used to build Windows. It
contains compilers and build tools. Although it comes with an
installer it's really just a tarball, you cam move it around. Many
shops keep it on a network share and many shops (Microsoft included)
keep it in they version control along with sources. Theoretically
it's offered to be used for writing drivers but it can build user mode
applications too (as I said, Windows itself is built with it). It has
the nice bonus of not coming with gigabytes of useless GUIs and also
the nice bonus of creating binaries that don't depend on Visual Studio
shared libraries.
> Not talking about the IDE. The last version
> of the Intel compiler I used was for 32-bit platforms and it required
> GCC when installed on OSX or Linux and Visual Studio's command line
> compiler tools on Windows.
No idea about OS X or Linux, on Windows it install a Visual Studio
plugin, hence the dependency, but it runs just fine without. In fact
Intel compiler's front-end come from EDG ( http://www.edg.com/ ), only
the back end is implemented in house.
--
Aram Hăvărneanu
Paulo Pinto
Joseph Poirier
>> To get the Visual Studio command line driven compiler tools you need
>> to install Visual Studio.
>
> No, you don't. Just get the WDK:
> http://msdn.microsoft.com/en-us/windows/hardware/gg487463
> It's a tarball of the build infrastructure used to build Windows. It
> contains compilers and build tools. Although it comes with an
> installer it's really just a tarball, you cam move it around. Many
> shops keep it on a network share and many shops (Microsoft included)
> keep it in they version control along with sources. Theoretically
> it's offered to be used for writing drivers but it can build user mode
> applications too (as I said, Windows itself is built with it). It has
> the nice bonus of not coming with gigabytes of useless GUIs and also
> the nice bonus of creating binaries that don't depend on Visual Studio
> shared libraries.
I agree. WDK and WinDBG were/are a great combination. At some point I
think WinDBG got packaged with the WDK.
Unfortunately, older versions of the Intel compiler (for 32-bit
platforms) wouldn't work properly unless it found Microsoft's command
line compiler tools via Visual Studio's installation directory
structure.
>
>> Not talking about the IDE. The last version
>> of the Intel compiler I used was for 32-bit platforms and it required
>> GCC when installed on OSX or Linux and Visual Studio's command line
>> compiler tools on Windows.
>
> No idea about OS X or Linux, on Windows it install a Visual Studio
> plugin, hence the dependency, but it runs just fine without. In fact
> Intel compiler's front-end come from EDG ( http://www.edg.com/ ), only
> the back end is implemented in house.
>
> --
> Aram Hăvărneanu
>
The Intel compiler has evolved a lot over the years, to include
removal of dependencies, and I'm not sure how it works these days.
Unfortunately it's C front-end still isn't fully C99 compliant.
-joe
Aram Hăvărneanu
> think WinDBG got packaged with the WDK.
Yes. It's good that's in the WDK, but it isn't so good that it's
*only* in the WDK and not available standalone. I used to install it
on all my computers. Good news is that it's still relocatable and if
you have a WDK installed somewhere you can steal WinDBG from there.
> Unfortunately it's C front-end still isn't fully C99 compliant.
On the other hand it's much more compliant than Visual Studio's
compiler. Microsoft was pretty clear that they won't do C99 and that
they expect people to write C++. Intel is at least trying to do C99.
--
Aram Hăvărneanu
Aram Hăvărneanu
And depending on the project it might work better or worse than the
WDK. It might work better because it comes with msbuild and all the
Visual Studio specific build tools so you might build Visual Studio
projects/solutions.
On the other hand you cannot maintain those solutions by hand because
they are a cluster of machine generated XML and the binaries depend on
Visual Studio shared libraries that have to be *installed* (by a .msi,
not copied along with the executable) on target computers.
WDK doesn't have msbuild, but it has build.exe (tool used to build
Windows). It's not a bad tool, you only need to create a makefile
that specifies the files and the target type. Sort of how Go
makefiles work. And binaries generated depend only on libraries that
are part of the operating system (you can target a wide variety of
older operating systems).
If someone would want to port some Unix software to Windows I'd
recommend the WDK, not Windows SDK or Visual Studio.
--
Aram Hăvărneanu
Remi Gillig
\Microsoft.NET) and you can use that only to build .sln files (but if
you have C++ projects you still need the Platform SDK).
Remi.