[PATCH] Makefile: support compressed debug info

[Nick Desaulniers]

As debug information gets larger and larger, it helps significantly save
the size of vmlinux images to compress the information in the debug
information sections. Note: this debug info is typically split off from
the final compressed kernel image, which is why vmlinux is what's used
in conjunction with GDB. Minimizing the debug info size should have no
impact on boot times, or final compressed kernel image size.

All of the debug sections will have a `C` flag set.
$ readelf -S <object file>

$ bloaty vmlinux.gcc75.compressed.dwarf4 -- \
vmlinux.gcc75.uncompressed.dwarf4

FILE SIZE VM SIZE
-------------- --------------
+0.0% +18 [ = ] 0 [Unmapped]
-73.3% -114Ki [ = ] 0 .debug_aranges
-76.2% -2.01Mi [ = ] 0 .debug_frame
-73.6% -2.89Mi [ = ] 0 .debug_str
-80.7% -4.66Mi [ = ] 0 .debug_abbrev
-82.9% -4.88Mi [ = ] 0 .debug_ranges
-70.5% -9.04Mi [ = ] 0 .debug_line
-79.3% -10.9Mi [ = ] 0 .debug_loc
-39.5% -88.6Mi [ = ] 0 .debug_info
-18.2% -123Mi [ = ] 0 TOTAL

$ bloaty vmlinux.clang11.compressed.dwarf4 -- \
vmlinux.clang11.uncompressed.dwarf4

FILE SIZE VM SIZE
-------------- --------------
+0.0% +23 [ = ] 0 [Unmapped]
-65.6% -871 [ = ] 0 .debug_aranges
-77.4% -1.84Mi [ = ] 0 .debug_frame
-82.9% -2.33Mi [ = ] 0 .debug_abbrev
-73.1% -2.43Mi [ = ] 0 .debug_str
-84.8% -3.07Mi [ = ] 0 .debug_ranges
-65.9% -8.62Mi [ = ] 0 .debug_line
-86.2% -40.0Mi [ = ] 0 .debug_loc
-42.0% -64.1Mi [ = ] 0 .debug_info
-22.1% -122Mi [ = ] 0 TOTAL

Suggested-by: David Blaikie <bla...@google.com>
Signed-off-by: Nick Desaulniers <nick.des...@gmail.com>
---
Makefile | 5 +++++
lib/Kconfig.debug | 9 +++++++++
2 files changed, 14 insertions(+)

diff --git a/Makefile b/Makefile
index 981eb902384b..313a054e5dc6 100644
--- a/Makefile
+++ b/Makefile
@@ -825,6 +825,11 @@ ifdef CONFIG_DEBUG_INFO_REDUCED
DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
$(call cc-option,-fno-var-tracking)
endif
+
+ifdef CONFIG_DEBUG_INFO_COMPRESSED
+DEBUG_CFLAGS += -gz=zlib
+KBUILD_LDFLAGS += --compress-debug-sections=zlib
+endif
endif

KBUILD_CFLAGS += $(DEBUG_CFLAGS)
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index f6f9a039f736..1f4a47ba6c1b 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -213,6 +213,15 @@ config DEBUG_INFO_REDUCED
DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.

+config DEBUG_INFO_COMPRESSED
+ bool "Compressed debugging information"
+ depends on DEBUG_INFO
+ depends on $(cc-option,-gz=zlib)
+ depends on $(ld-option,--compress-debug-sections=zlib)
+ help
+ Compress the debug information using zlib. Requires GCC 5.0+ or Clang
+ 5.0+.
+
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on DEBUG_INFO
--
2.17.1

[Sedat Dilek]

Hi Nick,

thanks for the patch.

I have slightly modified it to adapt to Linux v5.7-rc4 (what was your base?).

Which linker did you use and has it an impact if you switch from
ld.bfd to ld.lld?

I tried a first normal run and in a 2nd one with
CONFIG_DEBUG_INFO_COMPRESSED=y both with clang-10 and ld.lld-10.

My numbers (sizes in MiB):

[ diffconfig ]

$ scripts/diffconfig /boot/config-5.7.0-rc4-1-amd64-clang
/boot/config-5.7.0-rc4-2-amd64-clang
BUILD_SALT "5.7.0-rc4-1-amd64-clang" -> "5.7.0-rc4-2-amd64-clang"
+DEBUG_INFO_COMPRESSED y

[ compiler and linker ]

$ clang-10 -v
ClangBuiltLinux clang version 10.0.1
(https://github.com/llvm/llvm-project
92d5c1be9ee93850c0a8903f05f36a23ee835dc2)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/dileks/src/llvm-toolchain/install/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Candidate multilib: x32;@mx32
Selected multilib: .;@m64

$ ld.lld-10 -v
LLD 10.0.1 (https://github.com/llvm/llvm-project
92d5c1be9ee93850c0a8903f05f36a23ee835dc2) (compatible with GNU
linkers)

[ sizes vmlinux ]

$ du -m 5.7.0-rc4-*/vmlinux*
409 5.7.0-rc4-1-amd64-clang/vmlinux
7 5.7.0-rc4-1-amd64-clang/vmlinux.compressed
404 5.7.0-rc4-1-amd64-clang/vmlinux.o
324 5.7.0-rc4-2-amd64-clang/vmlinux
7 5.7.0-rc4-2-amd64-clang/vmlinux.compressed
299 5.7.0-rc4-2-amd64-clang/vmlinux.o

[ readelf (.debug_info as example) ]

$ readelf -S vmlinux.o
[33] .debug_info PROGBITS 0000000000000000 01d6a5e8
0000000006be1ee6 0000000000000000 0 0 1

$ readelf -S vmlinux.o
[33] .debug_info PROGBITS 0000000000000000 01749f18
0000000002ef04d2 0000000000000000 C 0 0 1 <---
XXX: "C (compressed)" Flag

Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
l (large), p (processor specific)

[ sizes linux-image debian packages ]

$ du -m 5.7.0-rc4-*/linux-image*.deb
47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb
424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb
47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb
771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb

[ sizes linux-git dir (compilation finished ]

5.7.0-rc4-1-amd64-clang: 17963 /home/dileks/src/linux-kernel/linux
5.7.0-rc4-2-amd64-clang: 14328 /home/dileks/src/linux-kernel/linux

[ xz compressed linux-image-dbg packages ]

$ file linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb
linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb:
Debian binary package (format 2.0), with control.tar.xz, data
compression xz
$ file linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb
linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb:
Debian binary package (format 2.0), with control.tar.xz, data
compression xz

[ file-lists ]

$ dpkg --contents
linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb
| wc -l
4395
$ dpkg --contents
linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb
| wc -l
4395

[ file-lists vmlinux ]

$ dpkg --contents
linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb
| grep vmlinux
-rwxr-xr-x root/root 428588312 2020-05-04 06:15
./usr/lib/debug/lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux
lrwxrwxrwx root/root 0 2020-05-04 06:15
./usr/lib/debug/boot/vmlinux-5.7.0-rc4-1-amd64-clang ->
../lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux
lrwxrwxrwx root/root 0 2020-05-04 06:15
./usr/lib/debug/vmlinux-5.7.0-rc4-1-amd64-clang ->
lib/modules/5.7.0-rc4-1-amd64-clang/vmlinux

$ dpkg --contents
linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb
| grep vmlinux
-rwxr-xr-x root/root 339341456 2020-05-04 12:24
./usr/lib/debug/lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux
lrwxrwxrwx root/root 0 2020-05-04 12:24
./usr/lib/debug/boot/vmlinux-5.7.0-rc4-2-amd64-clang ->
../lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux
lrwxrwxrwx root/root 0 2020-05-04 12:24
./usr/lib/debug/vmlinux-5.7.0-rc4-2-amd64-clang ->
lib/modules/5.7.0-rc4-2-amd64-clang/vmlinux

[ conclusion ]

As you can see there is a size-reduction in case of vmlinux/vmlinux.o
(debug) files...
...and my linux-git directory in total is smaller: 17963M vs. 14328M.

But the resulting linux-image-dbg file is much fatter: 424M vs. 711M.
XZ-compressing the gz/zlib-compressed vmlinux (debug) file results in
a fatter linux-image-dbg package.

For a usage of vmlinux (debug) in a VM scenario this might be nice but
seen from a debian repository perspective not.

For the sake of completeness: I have just installed and booted the
"normal" linux-image debian package - not the debug packages.

Thanks.

Regards,
- Sedat -

> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20200504031340.7103-1-nick.desaulniers%40gmail.com.

[Fangrui Song]

lld has supported the linker option --compress-debug-sections=zlib since
about 5.0.0 (https://reviews.llvm.org/D31941)

linker option --compress-debug-sections=zlib require binutils >= 2.26 (ld-option)
Assembler option --compress-debug-sections=zlib also require binutils >= 2.26 (cc-option,-gz=zlib)

>> config DEBUG_INFO_SPLIT
>> bool "Produce split debuginfo in .dwo files"
>> depends on DEBUG_INFO
>> --
>> 2.17.1
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/20200504031340.7103-1-nick.desaulniers%40gmail.com.
>
>--
>You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.

>To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CA%2BicZUUOaqeKeh6n4BJq2k6XQWAfNghUj57j42ZX5qyd3iOmLw%40mail.gmail.com.

[Masahiro Yamada]

Hi Sedat,

I also confirmed that, but this would not
be a blocker of this patch.

Users can disable CONFIG_DEBUG_INFO_COMPRESSED
if they care about the debug package size.





--
Best Regards
Masahiro Yamada

[Masahiro Yamada]

Nick,

I am OK with this patch.

Fangrui provided the minimal requirement for
--compress-debug-sections=zlib


Is it worth recording in the help text?
Do you want to send v2?

[Sedat Dilek]

Hi Masahiro,

No, it is not a blocker.
I have this patch now in my Linux v5.7-rc5 series.

I see a lot of more benefits concerning disc-usage - in my linux-git
and a reduced vmlinux file when I wanted to test with QEMU.

Feel free to add:
Tested-by: Sedat Dilek <sedat...@gmail.com>

- Sedat -

> Users can disable CONFIG_DEBUG_INFO_COMPRESSED
> if they care about the debug package size.
>
>
>
>
>
> --
> Best Regards
> Masahiro Yamada
>

> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAK7LNAR%2Bpm-_nd5%3DB2OeLpimW42FXxm8TQUMru9DR_asT3qYnA%40mail.gmail.com.

[Nick Desaulniers]

On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masa...@kernel.org> wrote:
>
> > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers
> > ><nick.des...@gmail.com> wrote:
> > >>
> > >> As debug information gets larger and larger, it helps significantly save
> > >> the size of vmlinux images to compress the information in the debug
> > >> information sections. Note: this debug info is typically split off from
> > >> the final compressed kernel image, which is why vmlinux is what's used
> > >> in conjunction with GDB. Minimizing the debug info size should have no
> > >> impact on boot times, or final compressed kernel image size.
> > >>

> Nick,
>
> I am OK with this patch.
>
> Fangrui provided the minimal requirement for
> --compress-debug-sections=zlib
>
>
> Is it worth recording in the help text?
> Do you want to send v2?

Yes I'd like to record that information. I can also record Sedat's
Tested-by tag. Thank you for testing Sedat.

I don't know what "linux-image-dbg file" are, or why they would be
bigger. The size of the debug info is the primary concern with this
config. It sounds like however that file is created might be
problematic.

Fangrui, I wasn't able to easily find what version of binutils first
added support. Can you please teach me how to fish?

Another question I had for Fangrui is, if the linker can compress
these sections, shouldn't we just have the linker do it, not the the
compiler and assembler? IIUC the debug info can contain relocations,
so the linker would have to decompress these, perform relocations,
then recompress these? I guess having the compiler and assembler
compress the debug info as well would minimize the size of the .o
files on disk.

Otherwise I should add this flag to the assembler invocation, too, in
v2. Thoughts?

I have a patch series that enables dwarf5 support in the kernel that
I'm working up to. I wanted to send this first. Both roughly reduce
the debug info size by 20% each, though I haven't measured them
together, yet. Requires ToT binutils because there have been many
fixes from reports of mine recently.
--
Thanks,
~Nick Desaulniers

[Fangrui Song]

On 2020-05-12, Nick Desaulniers wrote:
>On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masa...@kernel.org> wrote:
>>
>> > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers
>> > ><nick.des...@gmail.com> wrote:
>> > >>
>> > >> As debug information gets larger and larger, it helps significantly save
>> > >> the size of vmlinux images to compress the information in the debug
>> > >> information sections. Note: this debug info is typically split off from
>> > >> the final compressed kernel image, which is why vmlinux is what's used
>> > >> in conjunction with GDB. Minimizing the debug info size should have no
>> > >> impact on boot times, or final compressed kernel image size.
>> > >>
>> Nick,
>>
>> I am OK with this patch.
>>
>> Fangrui provided the minimal requirement for
>> --compress-debug-sections=zlib
>>
>>
>> Is it worth recording in the help text?
>> Do you want to send v2?
>
>Yes I'd like to record that information. I can also record Sedat's
>Tested-by tag. Thank you for testing Sedat.
>
>I don't know what "linux-image-dbg file" are, or why they would be
>bigger. The size of the debug info is the primary concern with this
>config. It sounds like however that file is created might be
>problematic.
>
>Fangrui, I wasn't able to easily find what version of binutils first
>added support. Can you please teach me how to fish?

I actually downloaded https://ftp.gnu.org/gnu/binutils/ archives and
located the sources... I think an easier way is:

% cd binutils-gdb
% git show binutils-2_26:./gas/as.c | grep compress-debug-sections
--compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\
...

GNU as 2.25 only supports --compress-debug-sections which means "zlib-gnu" in
newer versions.

Similarly, for GNU ld:

% git show binutils-2_26:./ld/lexsup.c | grep compress-debug-sections
--compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\

(I have spent a lot of time investigating GNU ld's behavior :)

>Another question I had for Fangrui is, if the linker can compress
>these sections, shouldn't we just have the linker do it, not the the
>compiler and assembler? IIUC the debug info can contain relocations,
>so the linker would have to decompress these, perform relocations,
>then recompress these? I guess having the compiler and assembler
>compress the debug info as well would minimize the size of the .o
>files on disk.

The linker will decompress debug info unconditionally. Because
input .debug_info sections need to be concatenated to form the output
.debug_info . Whether the output .debug_info is compressed is controlled
by the linker option --compress-debug-sections=zlib, which is not
affected by the compression state of object files.

Both GNU as and GNU ld name the option --compress-debug-sections=zlib.
In a compiler driver context, an unfamiliar user may find
-Wa,--compress-debug-sections=zlib -Wl,--compress-debug-sections=zlib
confusing:/

>Otherwise I should add this flag to the assembler invocation, too, in
>v2. Thoughts?

Compressing object files along with the linked output should be fine. It
can save disk space. (It'd be great if you paste the comparison
with and w/o object files compressed)

Feel free to add:

Reviewed-by: Fangrui Song <mas...@google.com>

>I have a patch series that enables dwarf5 support in the kernel that
>I'm working up to. I wanted to send this first. Both roughly reduce
>the debug info size by 20% each, though I haven't measured them
>together, yet. Requires ToT binutils because there have been many
>fixes from reports of mine recently.

This will be awesome! I also heard that enabling DWARF v5 for our object
files can easily make debug info size smaller by 20%. Glad that the
kernel can benefit it as well:)

[Sedat Dilek]

On Tue, May 12, 2020 at 9:23 PM Nick Desaulniers
<ndesau...@google.com> wrote:
>
> On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masa...@kernel.org> wrote:
> >
> > > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers
> > > ><nick.des...@gmail.com> wrote:
> > > >>
> > > >> As debug information gets larger and larger, it helps significantly save
> > > >> the size of vmlinux images to compress the information in the debug
> > > >> information sections. Note: this debug info is typically split off from
> > > >> the final compressed kernel image, which is why vmlinux is what's used
> > > >> in conjunction with GDB. Minimizing the debug info size should have no
> > > >> impact on boot times, or final compressed kernel image size.
> > > >>
> > Nick,
> >
> > I am OK with this patch.
> >
> > Fangrui provided the minimal requirement for
> > --compress-debug-sections=zlib
> >
> >
> > Is it worth recording in the help text?
> > Do you want to send v2?
>
> Yes I'd like to record that information. I can also record Sedat's
> Tested-by tag. Thank you for testing Sedat.
>
> I don't know what "linux-image-dbg file" are, or why they would be
> bigger. The size of the debug info is the primary concern with this
> config. It sounds like however that file is created might be
> problematic.
>

Hi Nick,

sorry, I try to explain the magic of "linux-image-dbg file".

In my workflow, I use the "scripts/package/{builddeb,mkdebian}"
shipped with the Linux-kernel as a base to build my Debian packages.

With enabled debugging (CONFIG_DEBUG_INFO=y) a
"linux-image-$KERNELRELEASE-dbg" Debian package is created.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/package/mkdebian#n203

As you can see below...

[ CONFIG_DEBUG_INFO_COMPRESSED=n ]

47 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang_5.7.0~rc4-1~bullseye+dileks1_amd64.deb
424 5.7.0-rc4-1-amd64-clang/linux-image-5.7.0-rc4-1-amd64-clang-dbg_5.7.0~rc4-1~bullseye+dileks1_amd64.deb

[ CONFIG_DEBUG_INFO_COMPRESSED=y ]

47 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang_5.7.0~rc4-2~bullseye+dileks1_amd64.deb
771 5.7.0-rc4-2-amd64-clang/linux-image-5.7.0-rc4-2-amd64-clang-dbg_5.7.0~rc4-2~bullseye+dileks1_amd64.deb

...there is minimal change in the size for the Debian package w/o
debug-infos - approx. 47M.
As said 424M vs. 771M for the dbg packages.

There is another big benefit checking my recorded stats:

$ grep 'cache size' stats/5.7.0-rc4-*/ccache-s.txt
stats/5.7.0-rc4-1-amd64-clang/ccache-s.txt:cache size
4.7 GB
stats/5.7.0-rc4-1-amd64-clang/ccache-s.txt:max cache size
10.0 GB
stats/5.7.0-rc4-2-amd64-clang/ccache-s.txt:cache size
3.4 GB
stats/5.7.0-rc4-2-amd64-clang/ccache-s.txt:max cache size
10.0 GB

So the cache of ccache is reduced: 4.7 GB vs. 3.4 GB

If you have any questions, "Don't ask to ask - just ask." :-).

[Sedat Dilek]

On Tue, May 12, 2020 at 10:01 PM Fangrui Song <mas...@google.com> wrote:
>
> On 2020-05-12, Nick Desaulniers wrote:

...

> >I have a patch series that enables dwarf5 support in the kernel that
> >I'm working up to. I wanted to send this first. Both roughly reduce
> >the debug info size by 20% each, though I haven't measured them
> >together, yet. Requires ToT binutils because there have been many
> >fixes from reports of mine recently.
>
> This will be awesome! I also heard that enabling DWARF v5 for our object
> files can easily make debug info size smaller by 20%. Glad that the
> kernel can benefit it as well:)

Please CC me on your pathcset - 20% smaller - I flip out.

- Sedat -

[Masahiro Yamada]

Nick,

On Wed, May 13, 2020 at 4:23 AM Nick Desaulniers
<ndesau...@google.com> wrote:
>
> On Mon, May 11, 2020 at 10:54 PM Masahiro Yamada <masa...@kernel.org> wrote:
> >
> > > >On Mon, May 4, 2020 at 5:13 AM Nick Desaulniers
> > > ><nick.des...@gmail.com> wrote:
> > > >>
> > > >> As debug information gets larger and larger, it helps significantly save
> > > >> the size of vmlinux images to compress the information in the debug
> > > >> information sections. Note: this debug info is typically split off from
> > > >> the final compressed kernel image, which is why vmlinux is what's used
> > > >> in conjunction with GDB. Minimizing the debug info size should have no
> > > >> impact on boot times, or final compressed kernel image size.
> > > >>
> > Nick,
> >
> > I am OK with this patch.
> >
> > Fangrui provided the minimal requirement for
> > --compress-debug-sections=zlib
> >
> >
> > Is it worth recording in the help text?
> > Do you want to send v2?
>
> Yes I'd like to record that information. I can also record Sedat's
> Tested-by tag. Thank you for testing Sedat.
>
> I don't know what "linux-image-dbg file" are, or why they would be
> bigger. The size of the debug info is the primary concern with this
> config. It sounds like however that file is created might be
> problematic.

As Sedat explained, deb package data
is compressed by xz, which is default.

You can use another compression method,
or disable compression if you desire.



"man dpkg-deb" says as follows:

-Zcompress-type
Specify which compression type to use when building a package.
Allowed values are gzip, xz (since dpkg 1.15.6), and none
(default is xz).



Kbuild supports KDEB_COMPRESS variable
to change the compression method.
See line 46 of scripts/package/builddeb.



If you are interested,
try "make bindeb-pkg" with/without CONFIG_DEBUG_INFO_COMPRESSED,
and compare the size of the generated debug package.




As Sedat stated,

(plain data) -> compress by gzip -> compress by xz

is often less efficient than

(plain data) -> compress by xz



I hope this is clearer.

> Fangrui, I wasn't able to easily find what version of binutils first
> added support. Can you please teach me how to fish?
>
> Another question I had for Fangrui is, if the linker can compress
> these sections, shouldn't we just have the linker do it, not the the
> compiler and assembler? IIUC the debug info can contain relocations,
> so the linker would have to decompress these, perform relocations,
> then recompress these? I guess having the compiler and assembler
> compress the debug info as well would minimize the size of the .o
> files on disk.
>
> Otherwise I should add this flag to the assembler invocation, too, in
> v2. Thoughts?
>
> I have a patch series that enables dwarf5 support in the kernel that
> I'm working up to. I wanted to send this first. Both roughly reduce
> the debug info size by 20% each, though I haven't measured them
> together, yet. Requires ToT binutils because there have been many
> fixes from reports of mine recently.
> --
> Thanks,
> ~Nick Desaulniers

[Sedat Dilek]

Hi Masahiro,

thanks for the clarification and the informations.

With next RC I will add KDEB_COMPRESS=gzip to my build-script and report.

Regards,
- Sedat -

[Nick Desaulniers]

On Tue, May 12, 2020 at 1:01 PM Fangrui Song <mas...@google.com> wrote:
>
> >Fangrui, I wasn't able to easily find what version of binutils first
> >added support. Can you please teach me how to fish?
>
> I actually downloaded https://ftp.gnu.org/gnu/binutils/ archives and
> located the sources... I think an easier way is:
>
> % cd binutils-gdb
> % git show binutils-2_26:./gas/as.c | grep compress-debug-sections

This assumes you knew to look at the binutils-2_26 tag, which is
putting the cart before the horse. ;)

I guess:
$ git log gas/as.c
/compress-debug-sections
commit 19a7fe52ae3d ("Make default compression gABI compliant")
looks related
$ git describe --contains "19a7fe52ae3d" | sed 's/~.*//'
users/hjl/linux/release/2.25.51.0.4
so it landed in 2.25.51.0.4.

+ Nick, H.J.
I'm unfamiliar with the git tag conventions of binutils. Does a patch
that landed in 2.25.51.0.4 mean it shipped in the official 2.25
release, or 2.26 release? Specifically, commit 19a7fe52ae3d.

> --compress-debug-sections[={none|zlib|zlib-gnu|zlib-gabi}]\n\
> ...
>
> GNU as 2.25 only supports --compress-debug-sections which means "zlib-gnu" in
> newer versions.
>
> Similarly, for GNU ld:
>
> % git show binutils-2_26:./ld/lexsup.c | grep compress-debug-sections
> --compress-debug-sections=[none|zlib|zlib-gnu|zlib-gabi]\n\
>
> (I have spent a lot of time investigating GNU ld's behavior :)
>
> >Another question I had for Fangrui is, if the linker can compress
> >these sections, shouldn't we just have the linker do it, not the the
> >compiler and assembler? IIUC the debug info can contain relocations,
> >so the linker would have to decompress these, perform relocations,
> >then recompress these? I guess having the compiler and assembler
> >compress the debug info as well would minimize the size of the .o
> >files on disk.
>
> The linker will decompress debug info unconditionally. Because
> input .debug_info sections need to be concatenated to form the output
> .debug_info . Whether the output .debug_info is compressed is controlled
> by the linker option --compress-debug-sections=zlib, which is not
> affected by the compression state of object files.
>
> Both GNU as and GNU ld name the option --compress-debug-sections=zlib.
> In a compiler driver context, an unfamiliar user may find
> -Wa,--compress-debug-sections=zlib -Wl,--compress-debug-sections=zlib
> confusing:/

The kernel uses the compiler as the driver for out of line assembly,
as they are all preprocessed first. Most out of line assembly in the
kernel uses the C preprocessor to #include headers that share #defines
of common constants shared between C and asm. #ifdef __ASSEMBLY__ is
used frequently in these headers. But for the linker, the linker
itself is invoked as the driver, though there are a few
inconsistencies we've cleaned up or still have to.

>
> >Otherwise I should add this flag to the assembler invocation, too, in
> >v2. Thoughts?
>
> Compressing object files along with the linked output should be fine. It
> can save disk space. (It'd be great if you paste the comparison
> with and w/o object files compressed)
>
> Feel free to add:
>
> Reviewed-by: Fangrui Song <mas...@google.com>

Thanks, will add that to v2.

>
> >I have a patch series that enables dwarf5 support in the kernel that
> >I'm working up to. I wanted to send this first. Both roughly reduce
> >the debug info size by 20% each, though I haven't measured them
> >together, yet. Requires ToT binutils because there have been many
> >fixes from reports of mine recently.
>
> This will be awesome! I also heard that enabling DWARF v5 for our object
> files can easily make debug info size smaller by 20%. Glad that the
> kernel can benefit it as well:)

--
Thanks,
~Nick Desaulniers

[Nick Clifton]

Hi Nick,

> + Nick, H.J.
> I'm unfamiliar with the git tag conventions of binutils. Does a patch
> that landed in 2.25.51.0.4 mean it shipped in the official 2.25
> release, or 2.26 release? Specifically, commit 19a7fe52ae3d.

2.26.

The convention is that a released form of the binutils has a version
number of X.XX or possible X.XX.N. The current mainline development
sources have a version of X.XX.50 where X.XX is the latest release.
(So the current mainline sources are version 2.34.50). When a release
happens the XX value is incremented by one as part of the release
process, and the .50 is dropped. (So the next binutils release will
be 2.35).

So 2.25.51.0.4 is a development version which will then have been
released as binutils 2.26.

Cheers
Nick

[Nick Desaulniers]

As debug information gets larger and larger, it helps significantly save
the size of vmlinux images to compress the information in the debug
information sections. Note: this debug info is typically split off from
the final compressed kernel image, which is why vmlinux is what's used
in conjunction with GDB. Minimizing the debug info size should have no
impact on boot times, or final compressed kernel image size.

For x86_64 defconfig + LLVM=1 (before):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.03
Maximum resident set size (kbytes): 43856

For x86_64 defconfig + LLVM=1 (after):
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:32.52
Maximum resident set size (kbytes): 1566776

Suggested-by: David Blaikie <bla...@google.com>
Suggested-by: Fangrui Song <mas...@google.com>
Suggested-by: Nick Clifton <ni...@redhat.com>
Suggested-by: Sedat Dilek <sedat...@gmail.com>
Tested-by: Sedat Dilek <sedat...@gmail.com>
Signed-off-by: Nick Desaulniers <ndesau...@google.com>
---
Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb
as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.

Makefile | 6 ++++++
lib/Kconfig.debug | 15 +++++++++++++++
2 files changed, 21 insertions(+)

diff --git a/Makefile b/Makefile
index 71687bfe1cd9..be8835296754 100644
--- a/Makefile
+++ b/Makefile
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
$(call cc-option,-fno-var-tracking)
endif

+ifdef CONFIG_DEBUG_INFO_COMPRESSED
+DEBUG_CFLAGS += -gz=zlib

+KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
+KBUILD_LDFLAGS += --compress-debug-sections=zlib
+endif
+
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
export DEBUG_CFLAGS

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index b8f023e054b9..5a423cbfaea4 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED

DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.

+config DEBUG_INFO_COMPRESSED
+ bool "Compressed debugging information"
+ depends on DEBUG_INFO
+ depends on $(cc-option,-gz=zlib)

+ depends on $(as-option,-Wa,--compress-debug-sections=zlib)

+ depends on $(ld-option,--compress-debug-sections=zlib)
+ help
+ Compress the debug information using zlib. Requires GCC 5.0+ or Clang

+ 5.0+, binutils 2.26+, and zlib.
+
+ Users of dpkg-deb via scripts/package/builddeb may
+ wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing
+ the debug info again with a different compression scheme, which can
+ result in larger binaries.
+

config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on DEBUG_INFO
--

2.26.2.761.g0e0b3e54be-goog

[Nick Desaulniers]

Sorry, I have mistyped David's email, it should be:
Suggested-by: David Blaikie <bla...@google.com>
(I missed the first `i` in the last name)

--
Thanks,
~Nick Desaulniers

[Andrew Morton]

I'm not sure who we're expecting to merge this, but I like shiny things ;)

huh, I didn't know kbuild/kconfig could do this. Does it all work as
expected when cross-compiling?

[Masahiro Yamada]

Suggested-by -> Reviewed-by

https://patchwork.kernel.org/patch/11524939/#23349551



> Suggested-by: Nick Clifton <ni...@redhat.com>


I do not know where this tag came from.

Nick Clifton taught us the version rule of binutils,but did not state
anything about this patch itself.

https://patchwork.kernel.org/patch/11524939/#23355175


> Suggested-by: Sedat Dilek <sedat...@gmail.com>

I do not see the source of this tag, either...

> Tested-by: Sedat Dilek <sedat...@gmail.com>
> Signed-off-by: Nick Desaulniers <ndesau...@google.com>
> ---

snip

> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED
> DEBUG_INFO build and compile times are reduced too.
> Only works with newer gcc versions.
>
> +config DEBUG_INFO_COMPRESSED
> + bool "Compressed debugging information"
> + depends on DEBUG_INFO
> + depends on $(cc-option,-gz=zlib)
> + depends on $(as-option,-Wa,--compress-debug-sections=zlib)

This does not work. (always false)
You cannot enable this option.

The comma between -Wa and --compress-debug-sections=zlib
is eaten by Kconfig parser because commas are delimiters
of function parameters.


Please write like this.

depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)

> + depends on $(ld-option,--compress-debug-sections=zlib)
> + help
> + Compress the debug information using zlib. Requires GCC 5.0+ or Clang
> + 5.0+, binutils 2.26+, and zlib.
> +
> + Users of dpkg-deb via scripts/package/builddeb may
> + wish to set the $KDEB_COMPRESS env var to "none" to avoid recompressing
> + the debug info again with a different compression scheme, which can
> + result in larger binaries.

No. This is not correct.

CONFIG_DEBUG_INFO_COMPRESSED compresses the only debug info part.
The other parts still get by benefit from the default KDEB_COMPRESS=xz.


The numbers are here:


CONFIG_DEBUG_INFO_COMPRESSED=y
-rw-r--r-- 1 masahiro masahiro 209077584 May 21 11:19
linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-26_amd64.deb


CONFIG_DEBUG_INFO_COMPRESSED=y and KDEB_COMPRESS=none
-rw-r--r-- 1 masahiro masahiro 643051712 May 21 11:22
linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-27_amd64.deb


CONFIG_DEBUG_INFO_COMPRESSED=n
-rw-r--r-- 1 masahiro masahiro 112200308 May 21 11:40
linux-image-5.7.0-rc5+-dbg_5.7.0-rc5+-30_amd64.deb




For the deb package size perspective,
it is better to keep KDEB_COMPRESS as default.

The main motivation for changing KDEB_COMPRESS
is the build speed. (see commit 1a7f0a34ea7d05)




CONFIG_DEBUG_INFO_COMPRESSED has a downside
for the debug deb package, but we need to accept it.

[Nick Desaulniers]

On Wed, May 20, 2020 at 7:48 PM Masahiro Yamada <masa...@kernel.org> wrote:
>
> > Suggested-by: Fangrui Song <mas...@google.com>
>
>
> Suggested-by -> Reviewed-by
>
> https://patchwork.kernel.org/patch/11524939/#23349551

Yes, my mistake.

> > Suggested-by: Nick Clifton <ni...@redhat.com>
>
>
> I do not know where this tag came from.
>
> Nick Clifton taught us the version rule of binutils,but did not state
> anything about this patch itself.
>
> https://patchwork.kernel.org/patch/11524939/#23355175
>
>
> > Suggested-by: Sedat Dilek <sedat...@gmail.com>
>
> I do not see the source of this tag, either...

Not all contributions to open source need to be in the form of
patches. Both Sedat and Nick gave suggestions which ultimately
informed the contents of this patch. They should be rewarded for
their efforts, and incentivized to help improve the code base further.
I think suggested by tags are a good way to do that; but if it's
against a written convention or if you still disagree, it's not the
end of the world to me, and you may drop those tags from the v3.

> > --- a/lib/Kconfig.debug
> > +++ b/lib/Kconfig.debug
> > @@ -225,6 +225,21 @@ config DEBUG_INFO_REDUCED
> > DEBUG_INFO build and compile times are reduced too.
> > Only works with newer gcc versions.
> >
> > +config DEBUG_INFO_COMPRESSED
> > + bool "Compressed debugging information"
> > + depends on DEBUG_INFO
> > + depends on $(cc-option,-gz=zlib)
> > + depends on $(as-option,-Wa,--compress-debug-sections=zlib)
>
> This does not work. (always false)

Technically, always true. `-Wa` disables all warnings from the
assembler. Also, I did test this via `make menuconfig`.

> You cannot enable this option.
>
> The comma between -Wa and --compress-debug-sections=zlib
> is eaten by Kconfig parser because commas are delimiters
> of function parameters.
>
>
> Please write like this.
>
> depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)

You're right, I knew this bug forgot. Will fix in v3.

Ah, I see. Thank you for those measurements. I'll send a v3 with
fixed up help text, but ultimately, I don't really care what it says
here. Please feel empowered to reword it should you choose to accept
+ apply it.
--
Thanks,
~Nick Desaulniers

[Nick Desaulniers]

Suggested-by: Nick Clifton <ni...@redhat.com>
Suggested-by: Sedat Dilek <sedat...@gmail.com>

Reviewed-by: Fangrui Song <mas...@google.com>

Tested-by: Sedat Dilek <sedat...@gmail.com>
Signed-off-by: Nick Desaulniers <ndesau...@google.com>
---

Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.

Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb
as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Makefile | 6 ++++++

lib/Kconfig.debug | 17 +++++++++++++++++
2 files changed, 23 insertions(+)

diff --git a/Makefile b/Makefile
index 71687bfe1cd9..be8835296754 100644

--- a/Makefile
+++ b/Makefile
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
$(call cc-option,-fno-var-tracking)
endif

+ifdef CONFIG_DEBUG_INFO_COMPRESSED
+DEBUG_CFLAGS += -gz=zlib
+KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
+KBUILD_LDFLAGS += --compress-debug-sections=zlib
+endif
+
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
export DEBUG_CFLAGS

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug

index b8f023e054b9..7fc82dcf814b 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED

DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.

+config DEBUG_INFO_COMPRESSED
+ bool "Compressed debugging information"
+ depends on DEBUG_INFO
+ depends on $(cc-option,-gz=zlib)

+ depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)

+ depends on $(ld-option,--compress-debug-sections=zlib)
+ help
+ Compress the debug information using zlib. Requires GCC 5.0+ or Clang
+ 5.0+, binutils 2.26+, and zlib.
+

+ Users of dpkg-deb via scripts/package/builddeb may find an increase in
+ size of their debug .deb packages with this config set, due to the
+ debug info being compressed with zlib, then the object files being
+ recompressed with a different compression scheme. But this is still
+ preferable to setting $KDEB_COMPRESS to "none" which would be even
+ larger.

+
config DEBUG_INFO_SPLIT
bool "Produce split debuginfo in .dwo files"
depends on DEBUG_INFO
--

2.27.0.rc0.183.gde8f92d652-goog

[Sedat Dilek]

Re-tested V3 on top of Linux v5.7-rc6+ with Clang/LLD v10.0.1-rc1.

- Sedat -

[Sedat Dilek]

On Sun, May 24, 2020 at 5:57 AM Masahiro Yamada <masa...@kernel.org> wrote:

>
> On Fri, May 22, 2020 at 6:57 AM 'Nick Desaulniers' via Clang Built
> Linux <clang-bu...@googlegroups.com> wrote:
> >
> > On Wed, May 20, 2020 at 7:48 PM Masahiro Yamada <masa...@kernel.org> wrote:
> > >
> > > > Suggested-by: Fangrui Song <mas...@google.com>
> > >
> > >
> > > Suggested-by -> Reviewed-by
> > >
> > > https://patchwork.kernel.org/patch/11524939/#23349551
> >
> > Yes, my mistake.
> >
> > > > Suggested-by: Nick Clifton <ni...@redhat.com>
> > >
> > >
> > > I do not know where this tag came from.
> > >
> > > Nick Clifton taught us the version rule of binutils,but did not state
> > > anything about this patch itself.
> > >
> > > https://patchwork.kernel.org/patch/11524939/#23355175
> > >
> > >
> > > > Suggested-by: Sedat Dilek <sedat...@gmail.com>
> > >
> > > I do not see the source of this tag, either...
> >
> > Not all contributions to open source need to be in the form of
> > patches. Both Sedat and Nick gave suggestions which ultimately
> > informed the contents of this patch. They should be rewarded for
> > their efforts, and incentivized to help improve the code base further.
> > I think suggested by tags are a good way to do that; but if it's
> > against a written convention or if you still disagree, it's not the
> > end of the world to me, and you may drop those tags from the v3.
>
>

> Documentation/process/submitting-patches.rst
> gives the guideline.
>
>
> "A Suggested-by: tag indicates that the patch idea is suggested by the person
> named and ensures credit to the person for the idea. Please note that this
> tag should not be added without the reporter's permission, especially if the
> idea was not posted in a public forum. That said, if we diligently credit our
> idea reporters, they will, hopefully, be inspired to help us again in the
> future."
>
>
> I think this tag should be given to people who
> gave the main idea to come up with
> the main part of the patch.
>
>
> Is that David Blaikie who suggested to
> compress the debug info ?
> If so, definitely he deserves to have Suggested-by tag.
>
> For the others, I do not think Suggested-by is a good fit.
>
> I appreciate their contribution to improve this patch.
> So, maybe you can give credit in other form, for example,
> mention it in the commit log explicitly?
>
> Nick Clifton helped us to provide the minimal binutils version.
> Sedat Dilet found an increase in size of debug .deb package.
>
>
> Thanks.

Hi,

first my last name is Dilek - just for the sake of completeness.
No, it is not my first name as Dilek is a female Turkish first name,
so I do not want to change my gender.

So this discussions come up again and again.

Thus some own words on this - this is my personal opinion.

Like the author of Curl and DOH said at FOSDEM 2019 in Bruessel:
I am doing all this work - first - for myself - in my build and
developing environment.
Very very egoistically!

"Share knowledge aggressively!"
...was Nick's words at First ClangBuiltLinux Meetup in Zurich 2020.
In a 2nd round I share my knowledge and I like this - that's why I am
doing Open Source.

For me it sounds like a "Suggested-by" tag or other credits like
"Reviewed-by" have a higher value than a Tested-by tag.

*** The opposite is the case. ***

Here, I am on a Samsung SandyBridge CPU/GPU aka 2nd generation
ultrabook series runing Debian/testing AMD64.

A slightly modified Debian-kernel linux-config takes me approx. 5 (in
words five) hours of compiling and generating Debian packages.

Plus, testing.
Plus, testing.
Plus, testing.

In Linux-next times I run the whole Linux-Test-Project tests plus some
FIO tests.

Finally, I decide depending from what is new and interesting to me to
attend a full single Linux-kernel release cycle.
The last was Linux v5.3 which was the first release to be
compile/link-able - with no modifications - with LLVM/Clang/LLD v9.0.
For upcoming Linux v5.7 I have built each single RC Linux-kernel and
used it in my daily work!
Since RC1 - for me running on bare metal counts - checking QEMU or
other VM is nice - but showed me that says sometimes nothing.

Plus, I am building llvm-toolchains (LLVM/Clang/LLD) and testing with
them (and report if needed).

"...if we diligently credit our idea reporters, they will, hopefully,
be inspired to help us again in the future."

These are some motivating words...

My Tested-by is like a certificate - like a "Made in Germany" seal :-).

Virtual Greeting from North-West Germany,
- Sedat -

[Masahiro Yamada]

On Fri, May 22, 2020 at 6:57 AM 'Nick Desaulniers' via Clang Built
Linux <clang-bu...@googlegroups.com> wrote:
>
> On Wed, May 20, 2020 at 7:48 PM Masahiro Yamada <masa...@kernel.org> wrote:
> >
> > > Suggested-by: Fangrui Song <mas...@google.com>
> >
> >
> > Suggested-by -> Reviewed-by
> >
> > https://patchwork.kernel.org/patch/11524939/#23349551
>
> Yes, my mistake.
>
> > > Suggested-by: Nick Clifton <ni...@redhat.com>
> >
> >
> > I do not know where this tag came from.
> >
> > Nick Clifton taught us the version rule of binutils,but did not state
> > anything about this patch itself.
> >
> > https://patchwork.kernel.org/patch/11524939/#23355175
> >
> >
> > > Suggested-by: Sedat Dilek <sedat...@gmail.com>
> >
> > I do not see the source of this tag, either...
>
> Not all contributions to open source need to be in the form of
> patches. Both Sedat and Nick gave suggestions which ultimately
> informed the contents of this patch. They should be rewarded for
> their efforts, and incentivized to help improve the code base further.
> I think suggested by tags are a good way to do that; but if it's
> against a written convention or if you still disagree, it's not the
> end of the world to me, and you may drop those tags from the v3.

Documentation/process/submitting-patches.rst
gives the guideline.


"A Suggested-by: tag indicates that the patch idea is suggested by the person
named and ensures credit to the person for the idea. Please note that this
tag should not be added without the reporter's permission, especially if the
idea was not posted in a public forum. That said, if we diligently credit our
idea reporters, they will, hopefully, be inspired to help us again in the
future."


I think this tag should be given to people who
gave the main idea to come up with
the main part of the patch.


Is that David Blaikie who suggested to
compress the debug info ?
If so, definitely he deserves to have Suggested-by tag.

For the others, I do not think Suggested-by is a good fit.

I appreciate their contribution to improve this patch.
So, maybe you can give credit in other form, for example,
mention it in the commit log explicitly?

Nick Clifton helped us to provide the minimal binutils version.
Sedat Dilet found an increase in size of debug .deb package.


Thanks.

>

> --
> You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-li...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/CAKwvOd%3DjOr4ZaLx-dSNTqZnGRATY1PZktUfu4JGWKRwRH%3DUjnw%40mail.gmail.com.

[Masahiro Yamada]

Suggested-by seems strange to me, but
I decided to not be worried too much.

Applied to linux-kbuild.

Thanks.

[Nick Desaulniers]

On Tue, May 26, 2020 at 3:28 AM Masahiro Yamada <masa...@kernel.org> wrote:
>
> > Suggested-by: David Blaikie <bla...@google.com>
> > Suggested-by: Nick Clifton <ni...@redhat.com>
> > Suggested-by: Sedat Dilek <sedat...@gmail.com>
> > Reviewed-by: Fangrui Song <mas...@google.com>
> > Tested-by: Sedat Dilek <sedat...@gmail.com>
> > Signed-off-by: Nick Desaulniers <ndesau...@google.com>
>
>
>
> Suggested-by seems strange to me, but
> I decided to not be worried too much.

Ah, sorry, Monday was a holiday. I unplugged for the long weekend.
https://en.wikipedia.org/wiki/Memorial_Day

I like the suggestion to simply say "thanks to ..." in the commit
message and will use that next time. I was ok to send a v4 with it.

>
> Applied to linux-kbuild.

Appreciated. I will have a dwarf-5 patch set, too. I'm not thrilled
with how I wired up Kconfig; maybe posting it for feedback is better
than me worrying about it too much. Hopefully will send this week,
assuming there's not too many bugs that require my immediate attention
(there never is...).

--
Thanks,
~Nick Desaulniers

[Nick Desaulniers]

On Sun, May 24, 2020 at 12:48 AM Sedat Dilek <sedat...@gmail.com> wrote:
>
> *** The opposite is the case. ***
>
> Here, I am on a Samsung SandyBridge CPU/GPU aka 2nd generation
> ultrabook series runing Debian/testing AMD64.
>
> A slightly modified Debian-kernel linux-config takes me approx. 5 (in
> words five) hours of compiling and generating Debian packages.
>
> Plus, testing.
> Plus, testing.
> Plus, testing.
>
> In Linux-next times I run the whole Linux-Test-Project tests plus some
> FIO tests.
>
> Finally, I decide depending from what is new and interesting to me to
> attend a full single Linux-kernel release cycle.
> The last was Linux v5.3 which was the first release to be
> compile/link-able - with no modifications - with LLVM/Clang/LLD v9.0.
> For upcoming Linux v5.7 I have built each single RC Linux-kernel and
> used it in my daily work!
> Since RC1 - for me running on bare metal counts - checking QEMU or
> other VM is nice - but showed me that says sometimes nothing.
>
> Plus, I am building llvm-toolchains (LLVM/Clang/LLD) and testing with
> them (and report if needed).

This is a lot of invaluable work. It means the world to me Sedat!

>
> "...if we diligently credit our idea reporters, they will, hopefully,
> be inspired to help us again in the future."
>
> These are some motivating words...
>
> My Tested-by is like a certificate - like a "Made in Germany" seal :-).

I love this, it is. Maybe if folks on this thread are bored, they
could help me with a personal project of mine?
https://github.com/nickdesaulniers/What-Open-Source-Means-To-Me
We could use more German, and Japanese.

>
> Virtual Greeting from North-West Germany,
> - Sedat -

--
Thanks,
~Nick Desaulniers

[Masahiro Yamada]

On Wed, May 27, 2020 at 12:53 AM 'Nick Desaulniers' via Clang Built
Linux <clang-bu...@googlegroups.com> wrote:
>
> On Tue, May 26, 2020 at 3:28 AM Masahiro Yamada <masa...@kernel.org> wrote:
> >
> > > Suggested-by: David Blaikie <bla...@google.com>
> > > Suggested-by: Nick Clifton <ni...@redhat.com>
> > > Suggested-by: Sedat Dilek <sedat...@gmail.com>
> > > Reviewed-by: Fangrui Song <mas...@google.com>
> > > Tested-by: Sedat Dilek <sedat...@gmail.com>
> > > Signed-off-by: Nick Desaulniers <ndesau...@google.com>
> >
> >
> >
> > Suggested-by seems strange to me, but
> > I decided to not be worried too much.
>
> Ah, sorry, Monday was a holiday. I unplugged for the long weekend.
> https://en.wikipedia.org/wiki/Memorial_Day
>
> I like the suggestion to simply say "thanks to ..." in the commit
> message and will use that next time. I was ok to send a v4 with it.

If you send v4, I will replace.

[Nick Desaulniers]

Thanks to:

Nick Clifton helped us to provide the minimal binutils version.
Sedat Dilet found an increase in size of debug .deb package.

Cc: Nick Clifton <ni...@redhat.com>
Cc: Sedat Dilek <sedat...@gmail.com>
Suggested-by: David Blaikie <bla...@google.com>

Reviewed-by: Fangrui Song <mas...@google.com>
Tested-by: Sedat Dilek <sedat...@gmail.com>
Signed-off-by: Nick Desaulniers <ndesau...@google.com>

---
Changes V3 -> V4:
* Add thanks line to commit message as per Masahiro.
* Swap Sugguested-by to Cc for two lines in commit message, as per
Masahiro.

Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.

Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb
as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.

Makefile | 6 ++++++

arch/arm64/kernel/vdso32/Makefile | 2 +-
lib/Kconfig.debug | 17 +++++++++++++++++
3 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 71687bfe1cd9..be8835296754 100644
--- a/Makefile
+++ b/Makefile
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
$(call cc-option,-fno-var-tracking)
endif

+ifdef CONFIG_DEBUG_INFO_COMPRESSED
+DEBUG_CFLAGS += -gz=zlib
+KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
+KBUILD_LDFLAGS += --compress-debug-sections=zlib
+endif
+
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
export DEBUG_CFLAGS

diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
index 3964738ebbde..5fd7792d03fc 100644
--- a/arch/arm64/kernel/vdso32/Makefile
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -135,7 +135,7 @@ c-obj-vdso-gettimeofday := vgettimeofday.o
asm-obj-vdso := sigreturn.o

ifneq ($(c-gettimeofday-y),)
-VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
+VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y) -marm
endif

VDSO_CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os

diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index b8f023e054b9..7fc82dcf814b 100644

--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -225,6 +225,23 @@ config DEBUG_INFO_REDUCED

DEBUG_INFO build and compile times are reduced too.
Only works with newer gcc versions.

+config DEBUG_INFO_COMPRESSED
+ bool "Compressed debugging information"
+ depends on DEBUG_INFO
+ depends on $(cc-option,-gz=zlib)

+ depends on $(as-option,-Wa$(comma)--compress-debug-sections=zlib)

+ depends on $(ld-option,--compress-debug-sections=zlib)
+ help
+ Compress the debug information using zlib. Requires GCC 5.0+ or Clang
+ 5.0+, binutils 2.26+, and zlib.
+

[Nick Desaulniers]

Sorry, I was wondering why Will and TGLX got cc'ed. My tree was dirty
when I amended ... was carrying another patch to send, please
disregard v4, and sorry for the noise.

--
Thanks,
~Nick Desaulniers

[Nick Desaulniers]

Reviewed-by: Fangrui Song <mas...@google.com>
Tested-by: Sedat Dilek <sedat...@gmail.com>
Signed-off-by: Nick Desaulniers <ndesau...@google.com>

---
Changes V4 -> V5:
* Drop unrelated hunk from a dirty tree.

Changes V3 -> V4:
* Add thanks line to commit message as per Masahiro.
* Swap Sugguested-by to Cc for two lines in commit message, as per
Masahiro.

Changes V2 -> V3:
* Fix blaikie@'s email addr.
* Fix Fangrui's Reviewed-by tag as per Masahiro.
* Fix help text as per Masahiro.
* Fix -Wa$(comma)foo as per Masahiro.

Changes V1 -> V2:
* rebase on linux-next.
* Add assembler flags as per Fangrui.
* Add note about KDEB_COMPRESS+scripts/package/builddeb
as per Sedat and Masahiro.
* Add note about bintutils version requirements as per Nick C.
* Add note about measured increased build time and max RSS.
Makefile | 6 ++++++

lib/Kconfig.debug | 17 +++++++++++++++++
2 files changed, 23 insertions(+)

diff --git a/Makefile b/Makefile
index 71687bfe1cd9..be8835296754 100644
--- a/Makefile
+++ b/Makefile
@@ -822,6 +822,12 @@ DEBUG_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \
$(call cc-option,-fno-var-tracking)
endif

+ifdef CONFIG_DEBUG_INFO_COMPRESSED
+DEBUG_CFLAGS += -gz=zlib
+KBUILD_AFLAGS += -Wa,--compress-debug-sections=zlib
+KBUILD_LDFLAGS += --compress-debug-sections=zlib
+endif
+
KBUILD_CFLAGS += $(DEBUG_CFLAGS)
export DEBUG_CFLAGS

[Sedat Dilek]

s/Dilet/Dilek

- Sedat -