What is content of a compressed section with ELFCOMPRESS_ZLIB?

[H.J. Lu]

The gABI says

ELFCOMPRESS_ZLIB
The section data is compressed with the ZLIB compression algorithm.
The compressed ZLIB data bytes begin with the byte immediately
following the compression header, and extend to the end of the
section. Additional documentation for ZLIB may be found at
http://zlib.net/.

My question is if the header, "ZLIB" + size (8 bytes in big endian),
is the part of the compressed ZLIB data bytes. I may have made a
mistake by including it in the compressed ZLIB data bytes.


--
H.J.

[Ali Bahrami]

Hi H.J.,

The sh_size in the section header, and the d_size
in the data descriptor, are the sum of the overhead
(the compression header: ZLIB + 8 in BE) and the number
of compressed bytes.

The size recorded in that compression header (ch_size for
the gABI format, "ZLIB" + 8 for the old one), are the
resulting value for sh_size and d_size when the data is
uncompressed. It doesn't include the size of the compression
header --- in the uncompressed state, there is no compression
header.

The decompress process, roughly, is:

- Verify the compression header, acquire the
uncompressed length.

- Allocate a replacement data buffer of that size,
and do the decompression.

- Update the data buffer (a_align, d_buf, d_size)
to use your new buffer.

- Update the section header sh_size. For gABI,
clear the SHF_COMPRESSED flag. For the old
style, deal with renaming the section from
.zdebug* to .debug*.

The way I think of this is that compression pushes the original
value of sh_size into the compression header, and decompression
pops it back.

- Ali

[H.J. Lu]

Hi Ali,

If I understand what you are saying correctly, a ELFCOMPRESS_ZLIB
section layout is

ELF compression header
The 12-byte overhead: "ZLIB" + 8 in BE
Compressed data

If it is true, why not skip the 12-byte, "ZLIB" + 8 in BE, overhead?
All the information is contained in ELF compression header.

Thanks.

--
H.J.

[Ali Bahrami]

On 05/14/15 08:32 AM, H.J. Lu wrote:
> Hi Ali,
>
> If I understand what you are saying correctly, a ELFCOMPRESS_ZLIB
> section layout is
>
> ELF compression header
> The 12-byte overhead: "ZLIB" + 8 in BE
> Compressed data
>
> If it is true, why not skip the 12-byte, "ZLIB" + 8 in BE, overhead?
> All the information is contained in ELF compression header.
>
> Thanks.
>

[Resending from the right email address so that it
reaches the list]

Hi H.J,

I misunderstood your question. When you said "ZLIB" + 8,
I assumed we were discussing the old GNU format, and I didn't
focus on the ELFCOMPRESS_ZLIB, which is of course the magic
number for the new gABI style.

Let me take another whack at it...

An ELFCOMPRESS_ZLIB section layout is

ELF compression header
Compressed data

There is no "ZLIB" + 8 in the gABI style. The Chdr compression
header takes the place of that old header, and supplies the same
information, in a more formal and extensible way (we can now add
new compression algorithms easily, not that I necessarily want
to see that).

Our docs for this:

http://docs.oracle.com/cd/E36784_01/html/E36857/section_compression.html

and

http://docs.oracle.com/cd/E36784_01/html/E36857/makehtml-id-7.html#scrolltoc

So yeah, you should leave those 12 byte out.
That's just for the old style.

- Ali