[llvm-dev] [LLD] Should --compress_debug_sections be enabled (=zlib) by default ?

[Romain GEISSLER via llvm-dev]

Hi,

In the file lld/ELF/Driver.cpp in function getCompressDebugSections we can see that the current default for lld is no debug section compression. It looks like tools like gdb, valgrind, elfutils, gcc's backtrace lib currently support compressed symbols. Since perf can use libdw from elfutils, I guess it supports it too.

Do you think it's time to enable compressed debug section by default ? On some of my big binaries (several hundreds of MB) this could save 50% of the overall binary size.

Cheers,
Romain
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[via llvm-dev]

> -----Original Message-----
> From: llvm-dev [mailto:llvm-dev...@lists.llvm.org] On Behalf Of
> Romain GEISSLER via llvm-dev
> Sent: Friday, May 03, 2019 12:53 PM
> To: llvm...@lists.llvm.org
> Subject: [llvm-dev] [LLD] Should --compress_debug_sections be enabled
> (=zlib) by default ?
>
> Hi,
>
> In the file lld/ELF/Driver.cpp in function getCompressDebugSections we can
> see that the current default for lld is no debug section compression. It
> looks like tools like gdb, valgrind, elfutils, gcc's backtrace lib
> currently support compressed symbols. Since perf can use libdw from
> elfutils, I guess it supports it too.
>
> Do you think it's time to enable compressed debug section by default ? On
> some of my big binaries (several hundreds of MB) this could save 50% of
> the overall binary size.

I think we'd want a way to make this customizable for different platforms.
--paulr

[Rui Ueyama via llvm-dev]

We can compress debug sections in compiler-generated object files as well as linker-generated executables. So the question whether we should compress debug sections is not only relevant to lld but to clang, and I think it's actually a whole toolchain-level decision.

If we decide to compress debug sections for some target, we should make a change to the compiler driver to flip the default for a compiler flag to turn on compression. lld is invoked by the compiler driver, so a change for the linker would be added to the compiler driver too.

I think I'm mildly in favor of making such change -- I guess the cost of compress doesn't too high, and that might even make things faster by reducing the amount of IO. I'd guess people wouldn't even care whether debug sections are compressed or not, as their tools already support it. But I don't have an evidence that the latter is actually the case.

[Romain GEISSLER via llvm-dev]

Le 7 mai 2019 à 09:03, Rui Ueyama <ru...@google.com> a écrit :

Hi,

We can compress debug sections in compiler-generated object files as well as linker-generated executables. So the question whether we should compress debug sections is not only relevant to lld but to clang, and I think it's actually a whole toolchain-level decision.

If we assume the compiler is clang. However in my case it would be gcc (I agree this is not a very ordinary mix of compiler/linker). When you use the full GNU toolchain, this choice of default value is done purely in binutils (you can configure it with the flag --enable-compressed-debug-sections={all,gas,gold,ld,none}).

Today as far as I know in binutils, to reduce IO, it's enabled by default in "gas" but not in "ld.bfd" (and by the way, it's not even implemented in "ld.gold", I have to submit a patch for that). This may be true for some target only, I am considering Linux x64. The fear in enabling it in "ld.bfd" is that it might break tools. For the tool I use (gdb, valgrind, (gcc) sanitizers (with gcc's libbacktrace for the stack printing)) it works. There might be other tools broken by this change. So yes it make sense to make the change at the whole toolchain level, in both the assembler + the linker. I don't know if clang uses it's own LLVM assembler or GNU gas.

I think I'm mildly in favor of making such change -- I guess the cost of compress doesn't too high, and that might even make things faster by reducing the amount of IO. I'd guess people wouldn't even care whether debug sections are compressed or not, as their tools already support it. But I don't have an evidence that the latter is actually the case.

Ok. I will just patch lld myself then, only for my own toolchain.

Thanks,

Romain

[Jan Kratochvil via llvm-dev]

On Fri, 03 May 2019 18:52:48 +0200, Romain GEISSLER via llvm-dev wrote:
> Do you think it's time to enable compressed debug section by default ?

It depends whether you use index (Apple index or .gdb_index or .debug_names).
If not then compressed debug is probably always a win.
.gdb_index is not going to be supported by LLDB (it is supported by GDB),
.debug_names (at LLD level) is going to be supported by LLDB in some time.

If you do use index then loading such file will be slower as this type of
compression cannot decompress only small section of the file addressed by the
index. Then some filesystem level (or block device level) compressed can be
a better compromise between disk size and debug info access performance.

But it sure needs more numbers.

Jan