RFC: Add STB_SECONDARY

[H.J.]

We want to provide a relocatable object which can take advantage of
all
versions of a supported OS. For a function, foo, in the C library, we
can use it only if it is available on all versions of the C library or
we provide our own implementation of foo. With our own foo, the one
in
the C library will never be used. Here is a proposal to add
STB_SECONDARY
to gABI to support the secondary definition so that a software vendor
can provide an alternative implementation in case it isn't available
in the C library.

STB_SECONDARY

Secondary symbols are similar to weak symbols, but their
definitions
have even lower precedence. Secondary symbols can only appear
in a
relocatable object. They must be either removed or converted to
global or local symbols once it has become part of an executable
or
shared object. The difference between secondary symbols and
weak
symbols are

1. The link editor must search archive library and extract
archive members to resolve defined and undefined secondary symbol.
2. When the link editor searches a shared object, it must honor
the global or weak definition in the shared object and ignore the
secondary one with the same name.
3. The link editor ignores the secondary definition if there is
a global, weak or common definition with the same name.
4. Unresolved secondary symbols have a zero value.

The purpose of this symbol binding is to provide the primary
definition as a global, weak or common symbol in an archive
library
or a shared object while keeping a secondary definition in a
relocatable object. If there is no primary definition, the
secondary definition will be used.

STB_SECONDARY is defined as:

#define STB_SECONDARY 3 /* Secondary symbol */

[Lowell, Randy]

Hi HJ,

What is the intended application of this? Are you planning to use this
for compiler-generated functions? I'm just wondering why you can't
accomplish what you've described here by just adding an archive library
to the end of the link line without introducing a new bind type.

Have you tested the behavior of existing linkers when they encounter
this new bind type? I tried it with our linker on HP-UX and it actually
handles the new bind type fairly well -- the STB_SECONDARY symbols are
silently ignored, but they do survive in the linked object's symbol table
as local symbols. This is better than I expected. Hopefully, other
linkers are as tolerant of the new bind type.

Is the new bind type exclusively for definitions? So STB_SECONDARY/SHN_UNDEF
is an invalid combination? Same for STB_SECONDARY/SHN_COMMON?

If you don't consider all the potential combinations and link behaviors when
the new bind type is added, implementers might define their own conflicting
behaviors.

Randy Lowell
HP

> --
> You received this message because you are subscribed to the Google
> Groups "Generic System V Application Binary Interface" group.
> To post to this group, send email to gener...@googlegroups.com.
> To unsubscribe from this group, send email to generic-
> abi+uns...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/generic-abi?hl=en.

[H.J. Lu]

On Mon, Apr 30, 2012 at 1:31 PM, Lowell, Randy <Randy....@hp.com> wrote:
> Hi HJ,
>
> What is the intended application of this?  Are you planning to use this
> for compiler-generated functions?  I'm just wondering why you can't

Yes.

> accomplish what you've described here by just adding an archive library
> to the end of the link line without introducing a new bind type.

It isn't transparent to users of our relocatable object files. They have
to manually place the alternative archive library after -lc, which normally
isn't specified at link command line.

> Have you tested the behavior of existing linkers when they encounter
> this new bind type?  I tried it with our linker on HP-UX and it actually

Not yet. If the existing linker doesn't handle STB_SECONDARY symbols
properly nor handle them like STB_GLOBAL symbols, compiler shouldn't
generate them.

> handles the new bind type fairly well -- the STB_SECONDARY symbols are
> silently ignored, but they do survive in the linked object's symbol table
> as local symbols.  This is better than I expected.  Hopefully, other
> linkers are as tolerant of the new bind type.
>
> Is the new bind type exclusively for definitions?  So STB_SECONDARY/SHN_UNDEF
> is an invalid combination?  Same for STB_SECONDARY/SHN_COMMON?

STB_SECONDARY/SHN_UNDEF is allowed, which is treated as
STB_GLOBAL/SHN_UNDEF if it can be resolved, otherwise, it is
treated as STB_WEAK/SHN_UNDEF. STB_SECONDARY/SHN_COMMON
isn't allowed.

> If you don't consider all the potential combinations and link behaviors when
> the new bind type is added, implementers might define their own conflicting
> behaviors.

That is a good point.

Thanks.

H.J.

> To unsubscribe from this group, send email to generic-abi...@googlegroups.com.

> For more options, visit this group at http://groups.google.com/group/generic-abi?hl=en.
>

--
H.J.

[Suprateeka R Hegde]

The requirement seems to indicate a user experience improvement
and not an ELF limitation. In this case, even if this new
binding is required, it could be a candidate to be added
under system specific semantics.

Now, this may need more clarity on the following:

1. If an app is fully archive bound (no dynamic linker)
the resolution should happen at link time. So the description
should include what happens in this case. Will the binding be
based on the OS and the library versions on the build machine
or the binding is based on the target run machine? Or is it
implicitly assumed that the app is going to run on the same
machine as it was built or the target machine's software version
is same as that of the build machine?

2. Take this requirement one more level deep in general.
Who is defined to be the winner when two STB_SECONDARY symbols
are found in the link? Is that order based? If so, then it again
becomes non-transparent to the user. The user has to fiddle with
ordering to get the right resolution. Or is this not required?

3. And then aren’t there tool chain specific approaches to provide
solution to this problem? Like emitting additional link time
information in new sections, etc.

--
Supra

> >> To post to this group, send email to generic-
> a...@googlegroups.com.

> >> To unsubscribe from this group, send email to generic-
> >> abi+uns...@googlegroups.com.
> >> For more options, visit this group at
> >> http://groups.google.com/group/generic-abi?hl=en.
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Generic System V Application Binary Interface" group.
> > To post to this group, send email to gener...@googlegroups.com.
> > To unsubscribe from this group, send email to generic-
> abi+uns...@googlegroups.com.
> > For more options, visit this group at
> http://groups.google.com/group/generic-abi?hl=en.
> >
>
>
>
> --

> H.J.

[H.J. Lu]

On Tue, May 1, 2012 at 8:20 AM, Suprateeka R Hegde
<hegdes...@gmail.com> wrote:
> The requirement seems to indicate a user experience improvement
> and not an ELF limitation. In this case, even if this new
> binding is required, it could be a candidate to be added
> under system specific semantics.
>
> Now, this may need more clarity on the following:
>
> 1. If an app is fully archive bound (no dynamic linker)
> the resolution should happen at link time. So the description
> should include what happens in this case. Will the binding be
> based on the OS and the library versions on the build machine
> or the binding is based on the target run machine? Or is it
> implicitly assumed that the app is going to run on the same
> machine as it was built or the target machine's software version
> is same as that of the build machine?

STB_SECONDARY tries to address a different problem
where the build machine of relocatable files is different from
the machine on which the relocatable files are used.
STB_SECONDARY will delay the binding until STB_SECONDARY
is consumed.

> 2. Take this requirement one more level deep in general.
> Who is defined to be the winner when two STB_SECONDARY symbols
> are found in the link? Is that order based? If so, then it again
> becomes non-transparent to the user. The user has to fiddle with
> ordering to get the right resolution. Or is this not required?

STB_SECONDARY is used to provide an alternative definition
when there is no primary one. Multiple secondary definitions with the
same name will not cause an error. The first appearance of the
secondary definition should be honored and the rest are ignored.

> 3. And then aren’t there tool chain specific approaches to provide
> solution to this problem? Like emitting additional link time
> information in new sections, etc.

STB_WEAK is the closest solution. But it doesn't work on Linux
when the primary definition is in libc.a/libc.so. We can use a
special section name. But it is easy to lead to conflicts.

H.J.

> To unsubscribe from this group, send email to generic-abi...@googlegroups.com.

[Lowell, Randy]

STB_WEAK is often used for one more "aliases" of an STB_GLOBAL
symbol. Would you define multiple STB_SECONDARY symbols with
the same address to be your alternate definitions to replace
these GLOBAL/WEAK pairings? Since the secondary is already
weaker than weak, I assume we would never need STB_WEAK_SECONDARY
or STB_TERTIARY?

> > 1. If an app is fully archive bound (no dynamic linker)
> > the resolution should happen at link time. So the description
> > should include what happens in this case. Will the binding be
> > based on the OS and the library versions on the build machine
> > or the binding is based on the target run machine? Or is it
> > implicitly assumed that the app is going to run on the same
> > machine as it was built or the target machine's software version
> > is same as that of the build machine?
>
> STB_SECONDARY tries to address a different problem
> where the build machine of relocatable files is different from
> the machine on which the relocatable files are used.
> STB_SECONDARY will delay the binding until STB_SECONDARY
> is consumed.

I know I mentioned this already, but since you're compiling these
objects on one system and consuming them on another system, it's
very important that either the default handling of unrecognized
symbol bind types is acceptable or there has to be some sort of
version checking happening at link time.

>
> > 2. Take this requirement one more level deep in general.
> > Who is defined to be the winner when two STB_SECONDARY symbols
> > are found in the link? Is that order based? If so, then it again
> > becomes non-transparent to the user. The user has to fiddle with
> > ordering to get the right resolution. Or is this not required?
>
> STB_SECONDARY is used to provide an alternative definition
> when there is no primary one. Multiple secondary definitions with the
> same name will not cause an error. The first appearance of the
> secondary definition should be honored and the rest are ignored.
>
> > 3. And then aren't there tool chain specific approaches to provide
> > solution to this problem? Like emitting additional link time
> > information in new sections, etc.
>
> STB_WEAK is the closest solution. But it doesn't work on Linux
> when the primary definition is in libc.a/libc.so. We can use a
> special section name. But it is easy to lead to conflicts.

A tool-specific approach could be semantically identical. It would
just be a bit messier to implement, but I'd rather see the ELF
structures being used as intended rather than burying symbol
resolution behavior in layers of obscurity.

Randy

[H.J. Lu]

On Tue, May 1, 2012 at 11:06 AM, Lowell, Randy <Randy....@hp.com> wrote:
> STB_WEAK is often used for one more "aliases" of an STB_GLOBAL
> symbol.  Would you define multiple STB_SECONDARY symbols with
> the same address to be your alternate definitions to replace
> these GLOBAL/WEAK pairings?   Since the secondary is already
> weaker than weak, I assume we would never need STB_WEAK_SECONDARY
> or STB_TERTIARY?

We don't need STB_WEAK_SECONDARY nor STB_TERTIARY.

>> > 1. If an app is fully archive bound (no dynamic linker)
>> > the resolution should happen at link time. So the description
>> > should include what happens in this case. Will the binding be
>> > based on the OS and the library versions on the build machine
>> > or the binding is based on the target run machine? Or is it
>> > implicitly assumed that the app is going to run on the same
>> > machine as it was built or the target machine's software version
>> > is same as that of the build machine?
>>
>> STB_SECONDARY tries to address a different problem
>> where the build machine of relocatable files is different from
>> the machine on which the relocatable files are used.
>> STB_SECONDARY will delay the binding until STB_SECONDARY
>> is consumed.
>
> I know I mentioned this already, but since you're compiling these
> objects on one system and consuming them on another system, it's
> very important that either the default handling of unrecognized
> symbol bind types is acceptable or there has to be some sort of
> version checking happening at link time.

STB_SECONDARY symbols should be generated only when the
consuming linker either understands STB_SECONDARY or treats
it as STB_GLOBAL/STB_WEAK.

H.J.
---

> To unsubscribe from this group, send email to generic-abi...@googlegroups.com.

[Suprateeka R Hegde]

On Tue, May 1, 2012 at 11:36 PM, Lowell, Randy <Randy....@hp.com> wrote:

STB_WEAK is often used for one more "aliases" of an STB_GLOBAL
symbol.  Would you define multiple STB_SECONDARY symbols with
the same address to be your alternate definitions to replace
these GLOBAL/WEAK pairings?   Since the secondary is already
weaker than weak, I assume we would never need STB_WEAK_SECONDARY
or STB_TERTIARY?

> > 1. If an app is fully archive bound (no dynamic linker)
> > the resolution should happen at link time. So the description
> > should include what happens in this case. Will the binding be
> > based on the OS and the library versions on the build machine
> > or the binding is based on the target run machine? Or is it
> > implicitly assumed that the app is going to run on the same
> > machine as it was built or the target machine's software version
> > is same as that of the build machine?
>
> STB_SECONDARY tries to address a different problem
> where the build machine of relocatable files is different from
> the machine on which the relocatable files are used.
> STB_SECONDARY will delay the binding until STB_SECONDARY
> is consumed.

What I meant by the build machine is the machine where these 
relocatable files are used in the link command to build the final
executable or shared library. I was not referring to the build
machine where these relocatable files are compiled.

Assume that the machine where these relocatable files
used on the link command does not have foo in libc.a and hence 
the STB_SECONDARY provided is consumed. Now what if the
machine where this executable is run, provides libc.a with foo?
In such case the semantics of STB_SECONDARY is not met.

 

I know I mentioned this already, but since you're compiling these
objects on one system and consuming them on another system, it's
very important that either the default handling of unrecognized
symbol bind types is acceptable or there has to be some sort of
version checking happening at link time.

>
> > 2. Take this requirement one more level deep in general.
> > Who is defined to be the winner when two STB_SECONDARY symbols
> > are found in the link? Is that order based? If so, then it again
> > becomes non-transparent to the user. The user has to fiddle with
> > ordering to get the right resolution. Or is this not required?
>
> STB_SECONDARY is used to provide an alternative definition
> when there is no primary one.  Multiple secondary definitions with the
> same name will not cause an error.  The first appearance of the
> secondary definition should be honored and the rest are ignored.

Similar to STB_WEAK with a difference that STB_SECONDARY is a
weaker WEAK required to improve user experience (get benefit without
changing the order of files on the link line).

This proposal targets only a very specific case of libc.a not providing the
symbol on some version of OS. Why not make it more generic?

Say:

ld 1.o 2.o lib1.a lib2.a libc.a

Now, assume that lib1.a is from a different vendor than lib2.a.
The vendor of lib2.a decides to provide a STB_SECONDARY 
foo to make the user avoid updating libc.a and without changing
the link order.

What if the vendor of lib1.a also wants to the do the same. The
vendor wants to provide a STB_SECONDARY foo only if there is
none in any of the link objects.

If we consider the generic case, the actual requirement then is
to have a mechanism to control the level of STB_WEAKness.

How about defining STB_SECONDARY as a symbol of bind
type STB_WEAK but wins only if it is the last one in the link order?

 

>
> > 3. And then aren't there tool chain specific approaches to provide
> > solution to this problem? Like emitting additional link time
> > information in new sections, etc.
>
> STB_WEAK is the closest solution.  But it doesn't work on Linux
> when the primary definition is in libc.a/libc.so.  We can use a
> special section name.  But it is easy to lead to conflicts.

A tool-specific approach could be semantically identical.  It would
just be a bit messier to implement, but I'd rather see the ELF
structures being used as intended rather than burying symbol
resolution behavior in layers of obscurity.

Alright. Then what about adding this under system specific semantics 

STB_LOOS through STB_HIOS? In that way gABI compliant linkers
are not forced to add the support for a user experience improvement.

Unless we are trying to provide a generic solution (and not just target
libc.a), a new standard binding type may not be substantial.

Also, the new standard type should change STB_NUM to 4.
#define STB_NUM 3 /* Number of defined types. */

However STB_NUM does not seem to be updated in the gABI 19 Oct 2010 draft.

--

Supra

Randy

To unsubscribe from this group, send email to generic-abi...@googlegroups.com.

[Ian Lance Taylor]

Suprateeka R Hegde <hegdes...@gmail.com> writes:

> Alright. Then what about adding this under system specific semantics
> STB_LOOS through STB_HIOS? In that way gABI compliant linkers
> are not forced to add the support for a user experience improvement.

H.J. originally proposed STB_GNU_SECONDARY = 11. But the GNU tools
already support STB_GNU_UNIQUE = 10, and that would leave us with only
one more avaliable OS-specific binding value, 12. We weren't convinced
that this functionality was valuable enough to burn one of our two
remaining binding values. The general ABI has 7 available binding
values, so there is a bit more room for something like this.

I think the link editor should treat STB_SECONDARY by recording the
definition and otherwise ignoring it. At the end of all symbol
processing, if there is a STB_SECONDARY definition for a symbol and no
other definition, then the link editor should resolve all references to
the symbol to the STB_SECONDARY definition. If there is another
definition, the STB_SECONDARY symbol should continue to be ignored. If
there are multiple STB_SECONDARY definitions, only record the first one
and ignore the subsequent ones.

The dynamic linker is a trickier case. I don't know if H.J. has thought
about it much. When the GNU dynamic linker sees a STB_WEAK definition
for a symbol, it does not search for other definitions for the symbol.
That is, if the executable has a STB_WEAK definition, the dynamic linker
does not search shared libraries. If some shared library has an
STB_WEAK definition, the dynamic linker does not search subsequent
shared libraries. This behaviour may be changed by setting the
undocumented environment variable LD_DYNAMIC_WEAK.

It is possible that the dynamic linker should handle STB_SECONDARY in
the same way. In that case, if the link editor chooses a STB_SECONDARY
definition for a symbol, it should probably set the binding to
STB_GLOBAL in the output executable or shared library.

Ian

[H.J. Lu]

I deliberately define STB_SECONDARY only for relocatable objects
so that no dynamic linker change is needed. STB_SECONDARY
should be converted to STB_GLOBAL or STB_LOCAL as needed.

H.J.

[H.J. Lu]

Since libc.a isn't used to run an executable, STB_SECONDARY
isn't relevant here. In my STB_SECONDARY proposal, all
STB_SECONDARY symbols are converted into STB_GLOBAL
or STB_LOCAL when they become parts of executable or
shared object.

>>
>>
>> I know I mentioned this already, but since you're compiling these
>> objects on one system and consuming them on another system, it's
>> very important that either the default handling of unrecognized
>> symbol bind types is acceptable or there has to be some sort of
>> version checking happening at link time.
>>
>> >
>> > > 2. Take this requirement one more level deep in general.
>> > > Who is defined to be the winner when two STB_SECONDARY symbols
>> > > are found in the link? Is that order based? If so, then it again
>> > > becomes non-transparent to the user. The user has to fiddle with
>> > > ordering to get the right resolution. Or is this not required?
>> >
>> > STB_SECONDARY is used to provide an alternative definition
>> > when there is no primary one.  Multiple secondary definitions with the
>> > same name will not cause an error.  The first appearance of the
>> > secondary definition should be honored and the rest are ignored.
>
>
> Similar to STB_WEAK with a difference that STB_SECONDARY is a
> weaker WEAK required to improve user experience (get benefit without
> changing the order of files on the link line).

That is correct.

> This proposal targets only a very specific case of libc.a not providing the
> symbol on some version of OS. Why not make it more generic?
>
> Say:
> ld 1.o 2.o lib1.a lib2.a libc.a
>
> Now, assume that lib1.a is from a different vendor than lib2.a.
> The vendor of lib2.a decides to provide a STB_SECONDARY
> foo to make the user avoid updating libc.a and without changing
> the link order.
>
> What if the vendor of lib1.a also wants to the do the same. The
> vendor wants to provide a STB_SECONDARY foo only if there is
> none in any of the link objects.

The first STB_SECONDARY foo will be used and linker will keep
searching for STB_GLOBAL/STB_WEAK foo.

> If we consider the generic case, the actual requirement then is
> to have a mechanism to control the level of STB_WEAKness.
>
> How about defining STB_SECONDARY as a symbol of bind
> type STB_WEAK but wins only if it is the last one in the link order?

That is basically my proposal.

>>
>> >
>> > > 3. And then aren't there tool chain specific approaches to provide
>> > > solution to this problem? Like emitting additional link time
>> > > information in new sections, etc.
>> >
>> > STB_WEAK is the closest solution.  But it doesn't work on Linux
>> > when the primary definition is in libc.a/libc.so.  We can use a
>> > special section name.  But it is easy to lead to conflicts.
>>
>> A tool-specific approach could be semantically identical.  It would
>> just be a bit messier to implement, but I'd rather see the ELF
>> structures being used as intended rather than burying symbol
>> resolution behavior in layers of obscurity.
>
>
> Alright. Then what about adding this under system specific semantics
> STB_LOOS through STB_HIOS? In that way gABI compliant linkers
> are not forced to add the support for a user experience improvement.

Ian has replied to this question.

> Unless we are trying to provide a generic solution (and not just target
> libc.a), a new standard binding type may not be substantial.
>
> Also, the new standard type should change STB_NUM to 4.
> #define STB_NUM 3 /* Number of defined types. */

STB_NUM will be updated if STB_SECODARY is accepted.

Thanks.

--
H.J.

[Suprateeka R Hegde]

> -----Original Message-----
> From: gener...@googlegroups.com [mailto:generic-
> a...@googlegroups.com] On Behalf Of H.J. Lu
> Sent: 02 May 2012 10:20 PM
> To: gener...@googlegroups.com
> Subject: Re: Add STB_SECONDARY
>

Right. That was clear in the proposal itself. My point here was:

Assume that the executable is built with the STB_SECONDARY def
because it was not available in libc.a at link time. So far so
fine. Now if the executable is run on a machine where libc.a
provides the def, then the executable still uses the STB_SECONDARY
def because it has been built fully archive.

In other words, we have a situation where the executable uses
STB_SECONDARY def even if the libc.a on the machine provides
the def.

Of course, this is more of a limitation of archive executable
and not the proposal. The proposal best fits the case where
the executable is run on the machine where it is linked. Only
then the semantics of STB_SECONDARY is meaningful.

> > This proposal targets only a very specific case of libc.a not
> providing the
> > symbol on some version of OS. Why not make it more generic?
> >
> > Say:
> > ld 1.o 2.o lib1.a lib2.a libc.a
> >
> > Now, assume that lib1.a is from a different vendor than lib2.a.
> > The vendor of lib2.a decides to provide a STB_SECONDARY
> > foo to make the user avoid updating libc.a and without changing
> > the link order.
> >
> > What if the vendor of lib1.a also wants to the do the same. The
> > vendor wants to provide a STB_SECONDARY foo only if there is
> > none in any of the link objects.
>
> The first STB_SECONDARY foo will be used and linker will keep
> searching for STB_GLOBAL/STB_WEAK foo.

What I am suggesting is to pick the last STB_SECONDARY foo.

In the above example, when both lib1.a and lib2.a say
"I want my def to be the last one to be picked", lets pick
from the last. The above is just an extended case of the original:

ld 1.o lib1.a libc.a

where lib1.a expects the def to be picked from libc.a if
available.

In that way, we can define the "winner STB_SECONDARY" as the
last of all definitions available on the link line.

What is the drawback of this approach?

>
> > If we consider the generic case, the actual requirement then is
> > to have a mechanism to control the level of STB_WEAKness.
> >
> > How about defining STB_SECONDARY as a symbol of bind
> > type STB_WEAK but wins only if it is the last one in the link
> order?
>
> That is basically my proposal.

The proposal says to pick the first one if there multiple
STB_SECONDARY. I am suggesting to use the last one.

> > Unless we are trying to provide a generic solution (and not just
> target
> > libc.a), a new standard binding type may not be substantial.
> >
> > Also, the new standard type should change STB_NUM to 4.
> > #define STB_NUM 3 /* Number of defined types. */
>
> STB_NUM will be updated if STB_SECODARY is accepted.

STB_SECONDARY is a useful addition to ELF, but my only concern
is that we are spending a standard value in ELF bind types
for a very corner case of libc.a It would have been better
if we can make it more useful by extending it to dynamic linker
processing at runtime.

-Supra

[Lowell, Randy]

I understand Supra's point here, but I have a different spin.
STB_SECONDARY means choose me if there are no definitions anywhere
on the link line. Since it's already order neutral with respect
to symbols of other bind types, there is no compelling reason to
choose any particular order with respect to other STB_SECONDARY
symbols. In all other cases where order comes into play, we
choose the first. This is the expected behavior, and doing anything
different would lead to confusion.

>
> >
> > > If we consider the generic case, the actual requirement then is
> > > to have a mechanism to control the level of STB_WEAKness.
> > >
> > > How about defining STB_SECONDARY as a symbol of bind
> > > type STB_WEAK but wins only if it is the last one in the link
> > order?
> >
> > That is basically my proposal.
>
> The proposal says to pick the first one if there multiple
> STB_SECONDARY. I am suggesting to use the last one.
>
> > > Unless we are trying to provide a generic solution (and not just
> > target
> > > libc.a), a new standard binding type may not be substantial.
> > >
> > > Also, the new standard type should change STB_NUM to 4.
> > > #define STB_NUM 3 /* Number of defined types. */
> >
> > STB_NUM will be updated if STB_SECODARY is accepted.
>
> STB_SECONDARY is a useful addition to ELF, but my only concern
> is that we are spending a standard value in ELF bind types
> for a very corner case of libc.a It would have been better
> if we can make it more useful by extending it to dynamic linker
> processing at runtime.

Yes. As it stands, this is only a partial solution, and it
precludes using the new bind type for dynamic symbol resolution.
I would also like to see a more complete proposal that allows
STB_SECONDARY to be used in dynamic symbol tables. The linker
should be allowed, but not required, to convert STB_SECONDARY
symbols to GLOBAL or LOCAL. The choice could then be controlled
with linker options.

Randy

>
> -Supra

[H.J. Lu]

Agree.

It will be nice to support STB_SECONDARY at run-time. When
resolving a symbol, after seeing a STB_SECONDARY definition,
dynamic linker should keep searching until a STB_GLOBAL or
STB_WEAK definition is found. If a STB_GLOBAL or STB_WEAK
definition is found, it will be used to satisfy the symbol lookup.
Otherwise, the STB_SECONDARY definition will be used. My
questions are

1. dlopen. What should happen when a shared object loaded
via dlopen contains a STB_GLOBAL definition with the same name
as a STB_SECONDARY definition in another shared object?
2. LD_PRELOAD. Should a STB_SECONADRY definition in
LD_PRELOAD object override a STB_GLOBAL definition?
3. Should it affect lazy bind?

--
H.J.

[Lowell, Randy]

> -----Original Message-----
> From: gener...@googlegroups.com [mailto:generic-
> a...@googlegroups.com] On Behalf Of H.J. Lu
> Sent: Thursday, May 03, 2012 8:28 PM
> To: gener...@googlegroups.com
> Subject: Re: Add STB_SECONDARY
>

Nothing special. Any symbol resolution done at dlopen time
follows the rules as at process startup. The module search
list used will be different, but resolving symbol refs and
symbol definition conflicts should be the same.

If the dlopen loads STB_GLOBAL foo after the program has
already resolved references to a STB_SECONDARY foo, those
references remain bound to the STB_SECONDARY symbol.

Any references to foo resolved after the dlopen, for which
the dlopen'd module is included in the module search list,
would be resolved to the STB_GLOBAL foo.

This is not unusual. Just one of many scenarios in which a
program can resolve references to multiple definitions of
the same symbol.

> 2. LD_PRELOAD. Should a STB_SECONADRY definition in
> LD_PRELOAD object override a STB_GLOBAL definition?

No. Don't change the rules for LD_PRELOAD.

> 3. Should it affect lazy bind?

Yes. Consistency is very important. It's easier to explain
how it works to users, and it's easier to defend when someone
claims that it's not working right.

Randy

>
> --
> H.J.

[H.J. Lu]

Here is the updated proposal:

We want to provide a relocatable object which can take advantage of all
versions of a supported OS. For a function, foo, in the C library, we
can use it only if it is available on all versions of the C library or
we provide our own implementation of foo. With our own foo, the one in
the C library will never be used. Here is a proposal to add STB_SECONDARY
to gABI to support the secondary definition so that a software vendor
can provide an alternative implementation in case it isn't available
in the C library.

STB_SECONDARY

Secondary symbols are similar to weak symbols, but their definitions

have even lower precedence. The difference between secondary symbols

and weak symbols are

1. The link editor must search archive library and extract
archive members to resolve defined and undefined secondary symbol.
2. When the link editor searches a shared object, it must honor
the global or weak definition in the shared object and ignore the
secondary one with the same name.
3. The link editor ignores the secondary definition if there is

a global, weak or common definition with the same name. Multiple

secondary definitions with the same name will not cause an error.
The first appearance of the secondary definition should be honored
and the rest are ignored.

4. The link editor treats the secondary definition in the shared
object as a global definition.
5. Unresolved secondary symbols have a zero value.

The purpose of this symbol binding is to provide the primary
definition as a global, weak or common symbol in an archive library
or a shared object while keeping a secondary definition in a
relocatable object. If there is no primary definition, the
secondary definition will be used.

When secondary definitions become part of an executable or shared
object, linker may convert them to global or local definitions.

At run-time, when resolving a symbol, after seeing a secondary
definition, the dynamic linker must keep searching until a
global or weak definition is found. If a global or weak

definition is found, it will be used to satisfy the symbol lookup.

Otherwise, the secondary definition will be used.

If the dlopen loads a global or weak definition after the program
has already resolved references to a secondary definition, those
references remain bound to the secondary definition. Any
references resolved after the dlopen, for which the dlopened

module is included in the module search list, would be resolved

to the global or weak definition.

[Suprateeka R Hegde]

Randy has replied taking few examples from lots of
possible symbol resolution scenarios. We should not
change any of the existing behavior.

A few points I would like to bring assuming it is not
implicit.

The original proposal said to mark STB_SECONDARY
as either global or local in the final executable/shlib.
Now that may need a change.

Assume that
ld 1.o lib1.a libc.so -o a.out

made a STB_SECONDARY foo from lib1.a
to be picked as libc.so didn’t have it
on the OS. Then in a.out, that becomes
a global or local which in other words
non-premptable by any STB_WEAK.

Assume that libc.so gets updated on the OS
and starts providing STB_WEAK foo, then in this
case the earlier linked executable still continues
to use the STB_SECONDARY which is not what we want.

I know that some platforms support an internal
version name for shlibs and hence the app continues
to use the old libc.so. But we are provding a more
generic solution applicable to all libraries
and not just libc. And not all libraries have version
associated with them.

In the above case, does it make sense to make the
static linker keep the STB_SECONDARY as is and let
the dynamic linker do its job?

Something similar to the existing technology as seen
in case of STT_COMMON:

http://www.sco.com/developers/gabi/latest/ch4.symtab.html#stt_common

Let the dynamic linker search for a non STB_SECONDARY def if
available.

Another point:
Assume that we have a dlopen(0,...) case which gives
a global handle for all load modules. Then dlsym on
such a handle should search entirely and not just
till STB_SECONDARY definition. In this case also,
the dynamic linker needs to know that the binding
type is STB_SECONDARY.

--
Supra

[H.J. Lu]

Please comment on my latest proposal to support STB_SECONDAY
at run-time..

Thanks.


--
H.J.

[Suprateeka R Hegde]

> Here is the updated proposal:

> 4. The link editor treats the secondary definition in the
> shared
> object as a global definition.

Assume that
ld 1.o lib1.so libc.so

makes link editor treat the secondary definition foo from
lib1.so as global, then even if libc.so provides one, the
symbol would be resolved to the one provided by lib1.so.

I assume that is true on all platforms?

> When secondary definitions become part of an executable or
> shared
> object, linker may convert them to global or local
> definitions.

So you are leaving it "implementation defined"? What is the
drawback in keeping STB_SECONDARY as is and not promote
to global/local atleast in shared mode?

Of course in fully archive mode, it makes sense to let the
linker promote secondary to global/local in executable.

> At run-time, when resolving a symbol, after seeing a secondary
> definition, the dynamic linker must keep searching until a
> global or weak definition is found. If a global or weak
> definition is found, it will be used to satisfy the symbol
> lookup.
> Otherwise, the secondary definition will be used.

Ok. this answers my thoughts I posted on my recent email. Our
emails crossed.

>
> If the dlopen loads a global or weak definition after the
> program
> has already resolved references to a secondary definition,
> those
> references remain bound to the secondary definition. Any
> references resolved after the dlopen, for which the dlopened
> module is included in the module search list, would be
> resolved
> to the global or weak definition.

Assume that we have a dlopen(0,...) case which gives

a global handle for all loaded modules. Then dlsym on

such a handle should search entirely and not just

till the first definition.

--
Supra

[Suprateeka R Hegde]

> -----Original Message-----
> From: Suprateeka R Hegde [mailto:hegdes...@gmail.com]
> Sent: 04 May 2012 11:12 PM
> To: 'gener...@googlegroups.com'
> Subject: RE: Add STB_SECONDARY
>

> > Here is the updated proposal:
>

> > 4. The link editor treats the secondary definition in the
> > shared
> > object as a global definition.
>

> Assume that
> ld 1.o lib1.so libc.so
>
> makes link editor treat the secondary definition foo from
> lib1.so as global, then even if libc.so provides one, the
> symbol would be resolved to the one provided by lib1.so.
>
> I assume that is true on all platforms?

I missed the point that I was thinking of the case when link
time optimizations or some sort of direct binding is in effect.

So it would be better to state that "the link editor *may* treat

the secondary definition in the shared object as a global definition.

--
Supra

[H.J. Lu]

On Fri, May 4, 2012 at 10:41 AM, Suprateeka R Hegde
<hegdes...@gmail.com> wrote:
>> Here is the updated proposal:
>
>>       4. The link editor treats the secondary definition in the
>> shared
>>       object as a global definition.
>
> Assume that
> ld 1.o lib1.so libc.so
>
> makes link editor treat the secondary definition foo from
> lib1.so as global, then even if libc.so provides one, the
> symbol would be resolved to the one provided by lib1.so.
>
> I assume that is true on all platforms?

Since STB_SECONDARY resolution is postponed to
run-time, it makes a little difference how secondary
definition in shared object is treated at link-time.
I can add "may" here.

>
>>       When secondary definitions become part of an executable or
>> shared
>>       object, linker may convert them to global or local
>> definitions.
>
> So you are leaving it "implementation defined"? What is the
> drawback in keeping STB_SECONDARY as is and not promote
> to global/local atleast in shared mode?

Make run-time STB_SECONDARY optional allows using
STB_SECONDARY in relocatable object without adding
STB_SECONDARY support to dynamic linker.

> Of course in fully archive mode, it makes sense to let the
> linker promote secondary to global/local in executable.
>
>>       At run-time, when resolving a symbol, after seeing a secondary
>>       definition, the dynamic linker must keep searching until a
>>       global or weak definition is found.  If a global or weak
>>       definition is found, it will be used to satisfy the symbol
>> lookup.
>>       Otherwise, the secondary definition will be used.
>
> Ok. this answers my thoughts I posted on my recent email. Our
> emails crossed.
>
>>
>>       If the dlopen loads a global or weak definition after the
>> program
>>       has already resolved references to a secondary definition,
>> those
>>       references remain bound to the secondary definition.  Any
>>       references resolved after the dlopen, for which the dlopened
>>       module is included in the module search list, would be
>> resolved
>>       to the global or weak definition.
>
> Assume that we have a dlopen(0,...) case which gives
> a global handle for all loaded modules. Then dlsym on
> such a handle should search entirely and not just
> till the first definition.
>

That is true.

--
H.J.

[Suprateeka R Hegde]

> Since STB_SECONDARY resolution is postponed to
> run-time, it makes a little difference how secondary
> definition in shared object is treated at link-time.
> I can add "may" here.

That "may" looks good.

> >> When secondary definitions become part of an executable or
> >> shared
> >> object, linker may convert them to global or local
> >> definitions.
> >
> > So you are leaving it "implementation defined"? What is the
> > drawback in keeping STB_SECONDARY as is and not promote
> > to global/local atleast in shared mode?
>
> Make run-time STB_SECONDARY optional allows using
> STB_SECONDARY in relocatable object without adding
> STB_SECONDARY support to dynamic linker.

Ok. that’s more flexible. Looks good.

Then we have a consensus on the proposal unless anybody
else have anything to edit.

--
Supra