Need to reconcile STV_ namespace in gABI

[Ali Bahrami]

Hi Cary and Everyone,

Now that we're about to have a living gABI again, I have
a piece of old unfinished business that I'd like to add to the
backlog for discussion.

Almost 13 years ago, in August of 2007, Rod made a proposal to this
list, to widen the visibility mask for st_other from 2 to 3 bits,
and to add new STV_ definitions. You can find that thread here:

https://groups.google.com/forum/#!topic/generic-abi/AahK3BVmYWQ

That thread didn't get wild support, nor angry rejection. Instead,
there's an amicable discussion about what it means, and of
alternatives, and then, of reasons not to chase those alternatives.
Finally, there was a wrap up that restated the proposal, and no
further discussion. The process for gABI additions was sort of fuzzy
in those days, so we thought that it had been accepted, and went
on to implement those changes. However, they didn't get added
to the gABI document, and so, we have a long standing divergence,
and the potential for a future conflict.

The Solaris <sys/elf.h> contains these details:

#define ELF32_ST_VISIBILITY(other) ((other)&0x7)
#define ELF64_ST_VISIBILITY(other) ((other)&0x7)

#define STV_DEFAULT 0
#define STV_INTERNAL 1
#define STV_HIDDEN 2
#define STV_PROTECTED 3
#define STV_EXPORTED 4
#define STV_SINGLETON 5
#define STV_ELIMINATE 6

#define STV_NUM 7

while <elf.h> on a Linux system still shows

#define ELF32_ST_VISIBILITY(o) ((o) & 0x03)

/* For ELF64 the definitions are the same. */
#define ELF64_ST_VISIBILITY(o) ELF32_ST_VISIBILITY (o)

/* Symbol visibility specification encoded in the st_other field. */
#define STV_DEFAULT 0 /* Default symbol visibility rules */
#define STV_INTERNAL 1 /* Processor specific hidden class */
#define STV_HIDDEN 2 /* Sym unavailable in other modules */
#define STV_PROTECTED 3 /* Not preemptible, not exported */

Unlike most of ELF, st_other has no OSABI or platform ranges, so
there's no OSABI area that could have been used instead, hence the
form of Rod's proposal. We would have made these OSABI specific
if that had been an option, and yet, I can't argue that st_other
should be carved up that way. The unused bits in st_other are
reserved against unknown future need, and there's just not a
lot of wasted space to play with. The real estate is just too
precious to carve up in this way. And so, we have to work out
these details, case by case. Thankfully, this isn't an area
where lots of changes occur.

I'd appreciate it if the gABI could recognize the existing
definitions from Rod's 2007 proposal. As Rod pointed out
back then, this is mainly a namespace issue. There's no
need for any other ELF implementations to implement them.
The primary goal is to reserve values 4-6 against being
assigned other meanings.

Thanks.

- Ali

[Alan Modra]

On Fri, Apr 17, 2020 at 11:23:56AM -0600, Ali Bahrami wrote:
> Unlike most of ELF, st_other has no OSABI or platform ranges, so
> there's no OSABI area that could have been used instead, hence the
> form of Rod's proposal. We would have made these OSABI specific
> if that had been an option, and yet, I can't argue that st_other
> should be carved up that way. The unused bits in st_other are
> reserved against unknown future need, and there's just not a
> lot of wasted space to play with. The real estate is just too
> precious to carve up in this way. And so, we have to work out
> these details, case by case. Thankfully, this isn't an area
> where lots of changes occur.

Despite the lack of formal OSABI or processor ranges in st_other, a
number of targets supported by GNU binutils steal some of the unused
bits. The PowerPC64 ELFv2 ABI uses the top three bits to describe
function entry requirements. AARCH64 uses the top bit. Alpha uses
the top bit of each nibble. M68HC12 uses the top bit. MIPS Irix uses
the top two bits. V850 uses the top 4 bits.

And SH uses bit 2, which clashes with the proposal to extend STV_.
(There's even a comment in include/elf/sh.h saying "Keep away from the
STV_ visibility flags" but whoever chose the value didn't keep away as
far as possible..)

--
Alan Modra
Australia Development Lab, IBM

[Ali Bahrami]

On 4/17/20 7:26 PM, Alan Modra wrote:
> Despite the lack of formal OSABI or processor ranges in st_other, a
> number of targets supported by GNU binutils steal some of the unused
> bits. The PowerPC64 ELFv2 ABI uses the top three bits to describe
> function entry requirements. AARCH64 uses the top bit. Alpha uses
> the top bit of each nibble. M68HC12 uses the top bit. MIPS Irix uses
> the top two bits. V850 uses the top 4 bits.
>
> And SH uses bit 2, which clashes with the proposal to extend STV_.
> (There's even a comment in include/elf/sh.h saying "Keep away from the
> STV_ visibility flags" but whoever chose the value didn't keep away as
> far as possible..)
>

Hi Alan,

Wow, that's worse than I had realized. It would seem that
every bit other than the least significant 2 have been used
by someone. And it's not going to be fast or productive
to try and move or change most of this use. I guess the future
might just be that we all go our own way with the top 6 bits
of st_other.

Other than SH, we might resolve this by reserving the top
4 or 5 bits to the "implementation", which I'd define as
the combination of OSABI and psABI. The gABI would cede those
bits. I say "4 or 5", because we might ask whether the Alpha
claim to the top bit of each nibble could be finessed somehow,
in order to claw back that precious extra bit.

That leaves SH as the show stopper. I did a quick search, but didn't
find the ABI. Can you point me at it, and what they use bit 2 for?
It probably can't be moved, but I'll ask the question, in case
there might be some way to finesse things.

Are there any SH folks on this list?

Thanks.

- Ali

[Fangrui Song]

Irix took the (1<<2) bit (STO_OPTIONAL). It seems that binutils merges the bit (https://sourceware.org/bugzilla/show_bug.cgi?id=1150) but ignores its semantics.

// bfd/elfxx-mips.c

          /* This is an optional symbol - an Irix specific extension to the

             ELF spec.  Ignore it for now.

             XXX - FIXME - there is more to the spec for OPTIONAL symbols

             than simply ignoring them, but we do not handle this for now.

             For information see the "64-bit ELF Object File Specification"

             which is available from here:

             http://techpubs.sgi.com/library/manuals/4000/007-4658-001/pdf/007-4658-001.pdf  */

          symbol = 0;

In LLVM, this bit is called STO_MIPS_OPTIONAL. The different name merely has exhibition difference. No LLVM tool implements its semantics.

Given that Irix is discontinued, we don't need to treat it as a show stopper.

[Alan Modra]

The Alpha binutils code modifies relocation processing on the values
0x80 and 0x88 in those two bits. A value of 0x08 is treated like a
value of 0. So it should be possible to use 0x08 for some other
purpose.

>
> That leaves SH as the show stopper. I did a quick search, but didn't
> find the ABI. Can you point me at it, and what they use bit 2 for?

I couldn't find a SuperH ABI myself. This comment describes its
effect:
/* A STO_SH5_ISA32 causes a "bitor 1" to the
symbol value, unless we've seen
STT_DATALABEL on the way to it. */

Hmm, the code is very likely dead since my removal of most sh5 and
sh64 code in binutils commit 211dc24b8744e. Those targets were marked
obsolete in commit 2b213129c51f.
https://sourceware.org/pipermail/binutils/2015-December/091090.html

> It probably can't be moved, but I'll ask the question, in case
> there might be some way to finesse things.
>
> Are there any SH folks on this list?
>
> Thanks.
>
> - Ali
>

> --
> You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/generic-abi/6baf29c1-dad9-6bc8-fc63-0fe3eef10586%40Oracle.COM.

[Ali Bahrami]

It seems that we still have the ability to expand the number
of bits used for STV values in st_other, despite having found
implementations using every one of the bits. Here is a table
summarizing the st_other use that we're aware of:

Bits Use
--------------------------------------------------------------------
0 gABI STV
1 gABI STV
2 Solaris STV, SH
3 Alpha
4 V850
5 PowerPC64, V850
6 PowerPC64, Irix, V850
7 PowerPC64, AARCH64, Alpha, M68HC12, Irix, V850

Making room means that we would choose to discount the claims
on bits 2 and 3 by implementations that are no longer current.

The most constraining claim was that of SH on bit 2. However, it
turns out that SH didn't get far enough for this assignment to
be a blocker. Alan pushed a fix to binutils yesterday that removed
the vestigial SH support:

https://sourceware.org/pipermail/binutils/2020-April/110762.html

with the comment

> SH-5 was stillborn. I don't see much point in reserving anything for
> it in binutils. See also https://lkml.org/lkml/2020/4/20/522

Alan also noted the following about bit 3, which is used by Alpha,
on this thread yesterday:

> The Alpha binutils code modifies relocation processing on the values
> 0x80 and 0x88 in those two bits. A value of 0x08 is treated like a
> value of 0. So it should be possible to use 0x08 for some other
> purpose.

And so, I'd like to ask for the following actions:

1) The gABI should adopt the widening of the STV
mask from 0x3 to 0x7, and reserve the 3 values
proposed by Rod in his original request.

2) The gABI should reserve bit 3 to itself today,
without defining any particular purpose for it.

The most likely eventual use for this would be to
allow the STV range to expand by one more bit, but
that decision can be left for later. In the meantime
(probably many years) Alpha would have time to migrate
off that bit. The existing conflict with Alpha will
need to be reevaluated if and when the gABI comes
to define a purpose for bit 3. By then, there may
no longer be a conflict. Failing that, the parties will
work out a solution at that time.

3) The gABI can't easily take back bits 4-7, so we
can continue to say nothing about them, or we can
explicitly reserve them to the implementation.

Thanks, and special thanks to Alan for doing all the shoveling.

- Ali