Re: ELF section flag SHF_EXCLUDE

[Mark Wielaard]

Hi,

Adding generic-abi to the CC to see what others think of SHF_EXCLUDE.

The below email from Alan Modra to the binutils mailinglist is from
last month. By accident SHF_EXCLUDE was made a generic section flag
about 10 years ago in binutils, even though it technically is in the
processor specific section flag space.

I recently stumbled over SHF_EXCLUDE again because GCC uses it for
certain LTO sections (and has been doing this since at least 2010). And
clang uses it for "single-file-split-DWARF". All usages are apparently
as if SHF_EXCLUDE is a generic section flag.

glibc elf.h has defined SHF_EXCLUDE since 2003, apparently because
Solaris defined it. I cannot find when it was introduced in Solaris.
The Solaris documentation describes it as:

SHF_EXCLUDE This section is excluded from input to the link-edit of
an executable or shared object. This flag is ignored if the
SHF_ALLOC flag is also set, or if relocations exist against the
section.

It doesn't call out whether it is processor specific or not.
binutils ld and gold don't check whether the SHF_ALLOC flag is set or
if there are relocations against the section, but simply drop it
always.

What do people think we should do about SHF_EXCLUDE?

Since it has been in use for more than 10 years now it seems best to
just declare it as a generic section flag, but it really isn't because
it uses a constant in the processor specific section numbers...

On Tue, 2020-03-10 at 22:19 +1030, Alan Modra wrote:
> commit 18ae9cc1db made SHF_EXCLUDE generic. Prior to that the flag
> was only defined for sparc, ppc, or32, and i370. The problem is that
> the flag was left in the very limited SHF_MASKPROC processor specific
> range rather than being put in the generic range, and it clashes with
> other processor specific section flags. aarch64, arm, hppa, mcore,
> microblaze, mips, mmix, nfp, score and v850 all define flags with the
> same value. This means SHF_EXCLUDE as is really shouldn't be used on
> any of those machines.
>
> aarch64.h:#define SHF_COMDEF 0x80000000
> arm.h:#define SHF_COMDEF 0x80000000
> hppa.h:#define SHF_PARISC_SBP 0x80000000
> mcore.h:#define SHF_MCORE_NOREAD 0x80000000
> microblaze.h:#define SHF_MICROBLAZE_NOREAD 0x80000000
> mips.h:#define SHF_MIPS_STRING 0x80000000
> mmix.h:#define SHF_MMIX_CANRELAX 0x80000000
> nfp.h:#define SHF_NFP_INIT 0x80000000
> score.h:#define SHF_SCORE_STRING 0x80000000
> v850.h:#define SHF_RENESAS_ABS 0x80000000
>
> SHF_PARISC_SBP, SHF_MCORE_NOREAD, SHF_MICROBLAZE_NOREAD,
> SHF_MIPS_STRING, SHF_MMIX_CANRELAX, SHF_SCORE_STRING, and
> SHF_RENESAS_ABS only appear in include/elf/*.h above. I haven't
> searched any of the relevant ABI docs, if such exist.
>
> SHF_COMDEF is decoded by readelf (once you disable SHF_EXCLUDE for
> arm
> and aarch64), but is described in arm docs by: "the legacy SHF_COMDEF
> ELF section flag is deprecated".
>
> SHF_NFP_INIT is used in opcodes/nfp-dis.c but not set by
> bfd/elf64-nfp.c.
>
> So what to do? Disabling the flag and assembler support for 'e' in
> flags is one possibility, but not a good idea for a 10 year old
> feature.
>

[Ali Bahrami]

On 4/16/20 10:38 AM, Mark Wielaard wrote:
> glibc elf.h has defined SHF_EXCLUDE since 2003, apparently because
> Solaris defined it. I cannot find when it was introduced in Solaris.
> The Solaris documentation describes it as:

Hi Mark,

Give me a day or so to do some digging and I'll
try to fill in some blanks.

This is what our <sys/elf.h> says:

/*
* The following processor specific flags have gained wide acceptance, and are
* implemented across all Solaris platforms.
*/
#define SHF_ORDERED 0x40000000 /* (EOL) See SHF_LINK_ORDER */
#define SHF_EXCLUDE 0x80000000

(Yes, irony: SHF_ORDERED "gained wide acceptance" for Sun
compilers, but it died out and was later nuked because the
gABI SHF_LINK_ORDER was better and simpler. History isn't
always pretty. :-) ).

So yes, this is another one of those cases where we
need to wrestle with whether it makes more sense to
define another generic value for this, or just accept that
all platforms would reserve that particular value for this
purpose.

-----
Thinking out loud here, and this is probably a bad idea...

Is it worth having a category of things that have become
so widespread that we just require all platforms, or OSABIs, to
reserve that value to that purpose? Then, a platform that doesn't
want it would be free to not use it, but we'd require them not
to use the value for any other purpose. The gABI document would
define these, and OS and platform ABIs would defer to it. The
numeric values would remain in the OS and platform ranges,

For instance, Solaris will probably not implement RELRO, but
it would be fine with us to reserve its value. In fact, our
header currently says:

/*
* Linux specific program headers not used by Solaris
*/
#define PT_GNU_STACK 0x6474e551 /* Indicates stack executability */
#define PT_GNU_RELRO 0x6474e552 /* Read-only after relocation */


There will be some clashes, where the value isn't available
everywhere, but in many cases, this would work. For instance:

#define SHT_SUNW_verdef 0x6ffffffd
#define SHT_GNU_verdef SHT_SUNW_verdef
#define SHT_SUNW_verneed 0x6ffffffe
#define SHT_GNU_verneed SHT_SUNW_verneed
#define SHT_SUNW_versym 0x6fffffff
#define SHT_GNU_versym SHT_SUNW_versym

I'm pretty sure there are non-GNUOSABI platforms using these.
At it seems simpler than having lots of things with both
generic, and non-generic, definitions, both of which need
to be supported.

My first reaction to stealing the psABI value for a new
SHF_UNWIND was the purist one of not wanting to do that, but
I'm concerned that if we start migrating a lot of these things
to new gABI values, we'll be creating a mess. Maybe it makes
sense to just say that all platforms are required to support
SHF_UNWIND with the value currently used by the SHF_86_64_UNWIND.
(I haven't searched for conflicts yet, so I'm making an
assumption here).

If so, we should move slowly and think very hard about how
it's managed under the gABI umbrella, because we'll only get
one shot at doing it well, and I'm sure there will be unanticipated
consequences.

- Ali

[Ali Bahrami]

On 4/16/20 10:38 AM, Mark Wielaard wrote:

> glibc elf.h has defined SHF_EXCLUDE since 2003, apparently because
> Solaris defined it. I cannot find when it was introduced in Solaris.
> The Solaris documentation describes it as:
>
> SHF_EXCLUDE This section is excluded from input to the link-edit of
> an executable or shared object. This flag is ignored if the
> SHF_ALLOC flag is also set, or if relocations exist against the
> section.

I expected this to take longer, but it's actually a pretty short
story. SHF_EXCLUDE was added to Solaris in 1996, in a PSARC case
entitled 'PSARC/1996/249 Addition of SHF_EXCLUDE Section Attribute Flag'.
The case was filed by my partner in crime of many years, Rod Evans.
And it was born as a psABI definition, for all our psABIs.

Here are the important details:

> Addition of a new attribute SHF_EXCLUDE
> =======================================
>
> Description
> -----------
> This is a proposal to add a definition for a new section header
> attribute, SHF_EXCLUDE, in the following header files:
> /usr/include/sys/elf_386.h
> /usr/include/sys/elf_SPARC.h
> /usr/include/sys/elf_ppc.h
> The SHF_EXCLUDE flag has the value 0x80000000
>
> Motivation
> ----------
> The motivation for this addition is to support the new section
> flag defined by PowerPC ABI.
>
> The new SHF_EXCLUDE flag is documented in:
> SYSTEM V APPILICATION BINARY INTERFACE
> PowerPC Processor Supplement
> September 1995
>
> The document describes the use of the flag as:
> The SHF_EXCLUDE flag specifies that the link editor is to
> exclude this section from executable and shared objects that
> it builds when those objects are not to be further relocated.
> SHF_EXCLUDE has the value 0x80000000.
>
> In addition to PowerPC requirement, and to be consistant with other
> ELF/link-editor feature, this attribute should be available on our
> other supported platforms.
>
> The linker needs to be modified to handle the sections
> with the new attribute on. Therefor the SHF_EXCLUDE flag
> has to be defined in a header file.

The requirement for "our other supported platforms" came from our
compiler group, which immediately found general use for it.

Hence, Sun added SHF_EXCLUDE in 1996, predating the gABI meetings
that happened in the early 2000's. It was born as a psABI value
on all the platforms of the time, and it seems that the reason
it was done as a psABI value rather than as a generic one is because
it was already in the PPC psABI, and we might not have been in
a position to have the PPC psABI changed.

I think it's probably also true that since this was before the
gABI meetings of the early 2000s, that having it be a psABI
value wasn't seen as being significantly different than as
a generic value, and so, didn't worry anyone (again, just guessing).
I don't know if SHF_EXCLUDE was discussed in those gABI meetings,
but SHF_EXCLUDE would have been in the Solaris headers at that
time, so it must have been at least seen by others at that time.

It's amusing to note that while we likely made this a psABI
value to accommodate the Power psABI, we only released one version
of Solaris for the PPC, Solaris 2.5.1. The PPC port was canceled
before the next (2.6) release.

Given the age and widespread use of this existing value on all
platforms, my preference would be that we continue with this
as is, and that new psABIs adopt SHF_EXCLUDE, or at least refrain
from assigning its value (0x80000000) for any other purpose.
Given that you're seeing widespread us of it in gcc and clang,
I think we'd be safe to formalize what already seems to be
a defacto standard.

- Ali