Should PIC in relocatable input change the behavior of output?

[H.J. Lu]

On x86, Linux linker generates outputs with different run-time behavior,
depending on if input files are compiled with PIC:

https://sourceware.org/ml/binutils/2016-02/msg00289.html

[hjl@gnu-tools-1 weak-1]$ cat x.c
extern void __attribute__ ((weak)) fun (void);
int main (void)
{
if (&fun != 0)
fun ();
return 0;
}
[hjl@gnu-tools-1 weak-1]$ make
readelf -r x

Relocation section '.rela.dyn' at offset 0x328 contains 1 entries:
Offset Info Type Sym. Value Sym. Name + Addend
000000600870 000200000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0

Relocation section '.rela.plt' at offset 0x340 contains 1 entries:
Offset Info Type Sym. Value Sym. Name + Addend
000000600890 000100000007 R_X86_64_JUMP_SLO 0000000000000000
__libc_start_main@GLIBC_2.2.5 + 0

There is no dynamic relocation against `fun'.

readelf -r y

Relocation section '.rela.dyn' at offset 0x348 contains 2 entries:
Offset Info Type Sym. Value Sym. Name + Addend
0000006008a0 000100000006 R_X86_64_GLOB_DAT 0000000000000000 fun + 0
0000006008a8 000300000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0

Relocation section '.rela.plt' at offset 0x378 contains 1 entries:
Offset Info Type Sym. Value Sym. Name + Addend
0000006008c8 000200000007 R_X86_64_JUMP_SLO 0000000000000000
__libc_start_main@GLIBC_2.2.5 + 0
[hjl@gnu-tools-1 weak-1]$

There is dynamic relocation against `fun'.

What does Solaris linker do?

--
H.J.

[Ali Bahrami]

Hi HJ,

I see that your x.c above corresponds to the undefweak.c
from the binutils thread, but I don't see a Makefile or
know how x and y were built.

If you'll send those extra bits, I'll be happy to reproduce
this for you and send you the output.

- Ali

[H.J. Lu]

Sorry. Here are the building steps:

[hjl@gnu-tools-1 weak-1]$ cat x.c
extern void __attribute__ ((weak)) fun (void);
int main (void)
{
if (&fun != 0)
fun ();
return 0;
}
[hjl@gnu-tools-1 weak-1]$ make

gcc -B./ -O2 -o x x.c
gcc -B./ -O2 -o y x.c -fPIC

readelf -r x

Relocation section '.rela.dyn' at offset 0x328 contains 1 entries:
Offset Info Type Sym. Value Sym. Name + Addend
000000600870 000200000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0

Relocation section '.rela.plt' at offset 0x340 contains 1 entries:
Offset Info Type Sym. Value Sym. Name + Addend
000000600890 000100000007 R_X86_64_JUMP_SLO 0000000000000000
__libc_start_main@GLIBC_2.2.5 + 0

readelf -r y

Relocation section '.rela.dyn' at offset 0x348 contains 2 entries:
Offset Info Type Sym. Value Sym. Name + Addend
0000006008a0 000100000006 R_X86_64_GLOB_DAT 0000000000000000 fun + 0
0000006008a8 000300000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0

Relocation section '.rela.plt' at offset 0x378 contains 1 entries:
Offset Info Type Sym. Value Sym. Name + Addend
0000006008c8 000200000007 R_X86_64_JUMP_SLO 0000000000000000
__libc_start_main@GLIBC_2.2.5 + 0
[hjl@gnu-tools-1 weak-1]$

What does Solaris linker do?

--
H.J.

[Michael Matz]

Hi,

On Sun, 21 Feb 2016, H.J. Lu wrote:

> On x86, Linux linker generates outputs with different run-time behavior,
> depending on if input files are compiled with PIC:
>
> https://sourceware.org/ml/binutils/2016-02/msg00289.html
>

> readelf -r y
>
> Relocation section '.rela.dyn' at offset 0x348 contains 2 entries:
> Offset Info Type Sym. Value Sym. Name + Addend
> 0000006008a0 000100000006 R_X86_64_GLOB_DAT 0000000000000000 fun + 0
> 0000006008a8 000300000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
>
> Relocation section '.rela.plt' at offset 0x378 contains 1 entries:
> Offset Info Type Sym. Value Sym. Name + Addend
> 0000006008c8 000200000007 R_X86_64_JUMP_SLO 0000000000000000
> __libc_start_main@GLIBC_2.2.5 + 0
> [hjl@gnu-tools-1 weak-1]$
>
> There is dynamic relocation against `fun'.
> What does Solaris linker do?

I don't know. But I'm fairly certain of the behaviour I would expect: PIC
or PIE or non-PIC/PIE should not change the behaviour in this case, and
the above relocations of 'y' are the correct ones (i.e. there should be a
dynamic reloc against 'fun' in all executables no matter how they were
produced, or on which target).


Ciao,
Michael.

[Suprateeka R Hegde]

I thought H.J wants the information only on Solaris/x86. So I replied to
him in private. since this is now open to all, I have the following
behavior from HP-UX on Intel Itanium.

Yes for this symbol 'fun', we generate dyn relocs.

However in the link phase of an executable, if there is no definition
available, this UNDEF WEAK is assigned a zero value. That means if
STT_OBJECT, it gets a storage with 0 as initialized value. If STT_FUNC,
the call to fun(), results in SEGV.

The above behavior is as per the gABI -- "Unresolved weak symbols have a
zero value".

At relocatable file stage or shared library stage, one cannot decide
that UNDEF WEAK will remain UNDEF even after final executable is
created. There might be some definition somewhere and binding for that
may needs to be patched. Hence relocs are necessary. But gABI does not
mandate this and says "implementation defined".

But support and compatibility is beyond the scope of gABI. So as Alan
said, relocs might be necessary.

--
Supra

>
>
> Ciao,
> Michael.
>

[H.J. Lu]

There are 2 issues:

1. Dynamic relocation against fun is missing in x, which isn't compiled
with PIC.
2. There is no dynamic relocation at all with static linking.

--
H.J.

[H.J. Lu]

So when building executable on HP-UX, undefined weak symbols
are always resolved to zero, regardless if input relocatable file is
compiled with PIC or not.


--
H.J.

[Michael Matz]

Hi,

On Mon, 22 Feb 2016, H.J. Lu wrote:

> >> Relocation section '.rela.dyn' at offset 0x348 contains 2 entries:
> >> Offset Info Type Sym. Value Sym. Name + Addend
> >> 0000006008a0 000100000006 R_X86_64_GLOB_DAT 0000000000000000 fun + 0
> >> 0000006008a8 000300000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
> >>
> >> Relocation section '.rela.plt' at offset 0x378 contains 1 entries:
> >> Offset Info Type Sym. Value Sym. Name + Addend
> >> 0000006008c8 000200000007 R_X86_64_JUMP_SLO 0000000000000000
> >> __libc_start_main@GLIBC_2.2.5 + 0
> >> [hjl@gnu-tools-1 weak-1]$
> >>
> >> There is dynamic relocation against `fun'.
> >> What does Solaris linker do?
> >
> > I don't know. But I'm fairly certain of the behaviour I would expect: PIC
> > or PIE or non-PIC/PIE should not change the behaviour in this case, and
> > the above relocations of 'y' are the correct ones (i.e. there should be a
> > dynamic reloc against 'fun' in all executables no matter how they were
> > produced, or on which target).
>
> There are 2 issues:
>
> 1. Dynamic relocation against fun is missing in x, which isn't compiled
> with PIC.

Yes, I'm aware of that, and I consider this a bug. The reloc should IMO
be there.

> 2. There is no dynamic relocation at all with static linking.

That's no problem. With static linking there are no dynamic libraries
loaded after the executable got created, so the weak reference can't
become non-NULL if it wasn't at link editing time. I ignore the problem
of dlopening shared libs from a static executable, that's not well
supported anyway.


Ciao,
Michael.

[Michael Matz]

Hi,

On Mon, 22 Feb 2016, Suprateeka R Hegde wrote:

> Yes for this symbol 'fun', we generate dyn relocs.

Um, so what is it? You say you do generate dyn relocs ...

> However in the link phase of an executable, if there is no definition
> available, this UNDEF WEAK is assigned a zero value. That means if
> STT_OBJECT, it gets a storage with 0 as initialized value. If STT_FUNC,
> the call to fun(), results in SEGV.

... but also say that you assign it a zero value, i.e. no dyn relocs.

Are you talking about the link editing phase (generating the executable
file) in the first paragraph, and the dynamic loading phase (generating
the memory image of the process) with no definition of 'fun' available in
the second?

(And I sure hope that the example program would not segfault on HP-UX, the
call to 'fun' is guarded by a null-pointer test).


Ciao,
Michael.

[H.J. Lu]

On Mon, Feb 22, 2016 at 7:45 AM, Michael Matz <ma...@suse.de> wrote:
> Hi,
>
> On Mon, 22 Feb 2016, H.J. Lu wrote:
>
>> >> Relocation section '.rela.dyn' at offset 0x348 contains 2 entries:
>> >> Offset Info Type Sym. Value Sym. Name + Addend
>> >> 0000006008a0 000100000006 R_X86_64_GLOB_DAT 0000000000000000 fun + 0
>> >> 0000006008a8 000300000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
>> >>
>> >> Relocation section '.rela.plt' at offset 0x378 contains 1 entries:
>> >> Offset Info Type Sym. Value Sym. Name + Addend
>> >> 0000006008c8 000200000007 R_X86_64_JUMP_SLO 0000000000000000
>> >> __libc_start_main@GLIBC_2.2.5 + 0
>> >> [hjl@gnu-tools-1 weak-1]$
>> >>
>> >> There is dynamic relocation against `fun'.
>> >> What does Solaris linker do?
>> >
>> > I don't know. But I'm fairly certain of the behaviour I would expect: PIC
>> > or PIE or non-PIC/PIE should not change the behaviour in this case, and
>> > the above relocations of 'y' are the correct ones (i.e. there should be a
>> > dynamic reloc against 'fun' in all executables no matter how they were
>> > produced, or on which target).
>>
>> There are 2 issues:
>>
>> 1. Dynamic relocation against fun is missing in x, which isn't compiled
>> with PIC.
>
> Yes, I'm aware of that, and I consider this a bug. The reloc should IMO
> be there.
>

gABI should be updated to clearly specify how undefined weak symbol
should be resolved in executable at link-time and we should change the
linker to follow the updated gABI.

--
H.J.

[Suprateeka R Hegde]

>>Undefined

>> The above behavior is as per the gABI -- "Unresolved weak symbols have a
>> zero value".
>>
>
> So when building executable on HP-UX, undefined weak symbols
> are always resolved to zero, regardless if input relocatable file is
> compiled with PIC or not.

Undefined weak symbols are always resolved to zero only if there is no
definition (either weak or global) seen by the linker during the
executable link. If there is a definition, then the UNDEF WEAK is
resolved to that definition.

If such a definition is in a shared library, then the executable will
have dynamic relocs to be processed by the dynamic linker at runtime. If
the definition is within another relocatable file, then obviously the
linker (ld) itself patches the references.

On HP-UX Itanium, we do not have non-PIC at all. It is all PIC. However,
on HP-UX PA-RISC, we have both PIC and non-PIC. And the behavior is as
described above irrespective of non-PIC or PIC.

--
Supra

[Michael Matz]

Maybe. Or we leave it implementation defined (or undefined?) in the gABI,
and define it reliably only in the GNU variant of it (gnu-gabi@ ! yeah! ;)
)


Ciao,
Michael.

[Suprateeka R Hegde]

Yes please. If I change the behavior now, it kills our compatibility
policy. I, in no way can afford to do that. We "must" make it part of
GNU-GABI and not SYSV gABI.

--
Supra

> )
>
>
> Ciao,
> Michael.
>

[H.J. Lu]

https://sourceware.org/ml/gnu-gabi/2016-q1/msg00004.html

--
H.J.

[Rod Evans]

On Sun, Feb 21, 2016 at 7:17 PM, H.J. Lu <hjl....@gmail.com> wrote:

What does Solaris linker do?

Is it really the linker?  I know we've seen instances where the
compiler has "optimized" away the test, concluding (I think) that
the call that follows means the function must exist.

We long ago advised folks to use dlsym() as a means of testing
for functionality, and to steer away from weak symbol reference
testing.

--

Rod.

[H.J. Lu]

On Mon, Feb 22, 2016 at 9:00 AM, Rod Evans <Rod....@oracle.com> wrote:
> On 2/22/16 4:28 AM, H.J. Lu wrote:
>> What does Solaris linker do?
>

> Is it really the linker? I know we've seen instances where the
> compiler has "optimized" away the test, concluding (I think) that
> the call that follows means the function must exist.

Compiler can't optimize it away due to

extern void __attribute__ ((weak)) fun (void);

> We long ago advised folks to use dlsym() as a means of testing
> for functionality, and to steer away from weak symbol reference
> testing.
>

This is a different issue. I'd like to know how Solaris linker
resolves reference to undefined weak symbol when creating
executable.


--
H.J.

[Ali Bahrami]

On 02/22/16 05:28, H.J. Lu wrote:
> What does Solaris linker do?
>

% cat x.c

extern void __attribute__ ((weak)) fun (void);
int main (void)
{
if (&fun != 0)
fun ();
return 0;
}

% gcc -B./ -O2 -o x x.c
% gcc -B./ -O2 -o y x.c -fPIC

% readelf -r x

Relocation section '.SUNW_reloc' at offset 0x830 contains 6 entries:
Offset Info Type Sym.Value Sym. Name
08060c24 00000106 R_386_GLOB_DAT 00000000 _Jv_RegisterClasses
08060c1c 00000306 R_386_GLOB_DAT 00000000 _ITM_deregisterTMClone
08060c20 00000806 R_386_GLOB_DAT 00000000 _ITM_registerTMCloneTa
08050b21 00000d01 R_386_32 00000000 fun
08050b30 00000d02 R_386_PC32 00000000 fun
08060c0c 00001306 R_386_GLOB_DAT 00000000 __start_crt_compiler

Relocation section '.rel.plt' at offset 0x860 contains 5 entries:
Offset Info Type Sym.Value Sym. Name
08060bf8 00000e07 R_386_JUMP_SLOT 08050898 exit
08060bfc 00001007 R_386_JUMP_SLOT 080508a8 _exit
08060c08 00000607 R_386_JUMP_SLOT 080508b8 atexit
08060c10 00001307 R_386_JUMP_SLOT 00000000 __start_crt_compiler
08060c18 00000707 R_386_JUMP_SLOT 080508d8 __fpstart


% readelf -r y

Relocation section '.SUNW_reloc' at offset 0x830 contains 5 entries:
Offset Info Type Sym.Value Sym. Name
08060c44 00000106 R_386_GLOB_DAT 00000000 _Jv_RegisterClasses
08060c3c 00000306 R_386_GLOB_DAT 00000000 _ITM_deregisterTMClone
08060c40 00000806 R_386_GLOB_DAT 00000000 _ITM_registerTMCloneTa
08060c48 00000d06 R_386_GLOB_DAT 00000000 fun
08060c2c 00001306 R_386_GLOB_DAT 00000000 __start_crt_compiler

Relocation section '.rel.plt' at offset 0x858 contains 6 entries:
Offset Info Type Sym.Value Sym. Name
08060c18 00000e07 R_386_JUMP_SLOT 08050898 exit
08060c1c 00001007 R_386_JUMP_SLOT 080508a8 _exit
08060c28 00000607 R_386_JUMP_SLOT 080508b8 atexit
08060c30 00001307 R_386_JUMP_SLOT 00000000 __start_crt_compiler
08060c38 00000707 R_386_JUMP_SLOT 080508d8 __fpstart
08060c4c 00000d07 R_386_JUMP_SLOT 00000000 fun

- Ali

[H.J. Lu]

Thanks. GNU linker should always generate dynamic relocation
against fun.


--
H.J.

[H.J. Lu]

On Mon, Feb 22, 2016 at 9:20 AM, Ali Bahrami <Ali.B...@oracle.com> wrote:

One more thing, does your linker generate DT_TEXTREL due to
R_386_32 relocation in .text?

--
H.J.

[Ali Bahrami]

Yes it does:

% elfdump -d x | grep TEXTREL
[16] TEXTREL 0
[24] FLAGS 0x4 [ TEXTREL ]
% elfdump -d y | grep TEXTREL

Our manual says:

https://docs.oracle.com/cd/E53394_01/html/E54813/chapter6-42444.html#scrolltoc

DT_TEXTREL
Indicates that one or more relocation entries might request
modifications to a non-writable segment, and the runtime linker
can prepare accordingly. This element's use has been superseded
by the DF_TEXTREL flag. See Position-Independent Code.

I wasn't around when that change happened, so I don't know
the history behind it. However, we create DT_TEXTREL as well
as setting DF_TEXTREL in DT_FLAGS. An oddity of DT_TEXTREL is
that the value doesn't mean anything --- here the object x has
relocations to text even though the value of DT_TEXTREL is 0.

- Ali

[H.J. Lu]

Linux x86 linker resolves undefined weak symbol to value 0 at link-time
when creating executable. It is in almost all programs on Linux/x86:

hjl@gnu-6 pr19704]$ readelf -sW /bin/true | grep WEAK | grep UND
15: 0000000000000000 0 NOTYPE WEAK DEFAULT UND
_ITM_deregisterTMCloneTable
70: 0000000000000000 0 NOTYPE WEAK DEFAULT UND __gmon_start__
102: 0000000000000000 0 NOTYPE WEAK DEFAULT UND _Jv_RegisterClasses
113: 0000000000000000 0 NOTYPE WEAK DEFAULT UND
_ITM_registerTMCloneTable
118: 0000000000000000 0 FUNC WEAK DEFAULT UND
__cxa_finalize@GLIBC_2.2.5 (3)

Changing Linux x86 linker to behave like Solaris linker means that all
Linux x86 binaries will have DT_TEXTREL. I don't think we want to
do that.

--
H.J.

[Rod Evans]

   DT_TEXTREL
        Indicates that one or more relocation entries might request
        modifications to a non-writable segment, and the runtime linker
        can prepare accordingly. This element's use has been superseded
        by the DF_TEXTREL flag. See Position-Independent Code.

I wasn't around when that change happened, so I don't know
the history behind it. However, we create DT_TEXTREL as well
as setting DF_TEXTREL in DT_FLAGS. An oddity of DT_TEXTREL is
that the value doesn't mean anything --- here the object x has
relocations to text even though the value of DT_TEXTREL is 0.

Its presence indicates text relocations remain.

Many compilers reference 'fun' as NOTY UNDEF.  If ld(1) does not resolve this

to a definition, then it doesn't know what to do.  If we bind to a function we

generate a .plt.  If we bind to a data object we generate a copy relocation.

But NOTY doesn't provide any info, hence a text relocation remains.  Another

reason why we recommend not using the address of a weak reference as

a test for functionality.

----
Rod.

[Ali Bahrami]

On 02/22/16 15:59, H.J. Lu wrote:
> Changing Linux x86 linker to behave like Solaris linker means that all
> Linux x86 binaries will have DT_TEXTREL. I don't think we want to
> do that.

As Rod pointed out in a parallel reply, we set TEXTREL
if there are text relocations, so it's purely informational.

My opinion is that no one wants text relocations, but not
labeling them if they do exist doesn't seem right. I find
the presence of DF_TEXTREL very interesting when diagnosing
various issues.

- Ali

[H.J. Lu]

Can you do the same experiment with non-PIC input on
x86-64 with Solaris linker?

Thanks.

--
H.J.

[Ali Bahrami]

On 02/24/16 17:22, H.J. Lu wrote:
>>> One more thing, does your linker generate DT_TEXTREL due to
>>> >>R_386_32 relocation in .text?
>>> >>
>> >
>> >
>> >Yes it does:
>> >
>> > % elfdump -d x | grep TEXTREL
>> > [16] TEXTREL 0
>> > [24] FLAGS 0x4 [ TEXTREL ]
>> > % elfdump -d y | grep TEXTREL
>> >

...

>> >
> Can you do the same experiment with non-PIC input on
> x86-64 with Solaris linker?

It's the same:

% gcc -m64 -B./ -O2 -o x64 x.c
% gcc -m64 -B./ -O2 -o y64 x.c -fPIC
% elfdump -d x64 | grep TEXTREL

[16] TEXTREL 0
[24] FLAGS 0x4 [ TEXTREL ]

% elfdump -d y64 | grep TEXTREL

and the readelf -r commands show relocations with the symbol
fun in both cases.

- Ali

[H.J. Lu]

I assume you have dynamic R_X86_64_32 relocation in .text
section. If I am correct, since R_X86_64_32 is 32-bit, you
will get run-time relocation overflow if the address of fun is above
4GB.


--
H.J.

[Ali Bahrami]

That's correct on both counts. If that became a problem, I
guess one could move to one of the larger code models, or
probably better, just use PIC.

I'll include the readelf output below.

- Ali

====================================================================================

% readelf -r x64

Relocation section '.SUNW_reloc' at offset 0xaf8 contains 6 entries:

Offset Info Type Sym. Value Sym. Name + Addend

000000500f98 000100000006 R_X86_64_GLOB_DAT 0000000000000000 _Jv_RegisterClasses + 0
000000500f88 000300000006 R_X86_64_GLOB_DAT 0000000000000000 _ITM_deregisterTMClone + 0
000000500f90 000800000006 R_X86_64_GLOB_DAT 0000000000000000 _ITM_registerTMCloneTa + 0
000000400e71 000d0000000a R_X86_64_32 0000000000000000 fun + 0
000000400e7f 000d00000002 R_X86_64_PC32 0000000000000000 fun - 4
000000500f68 001300000006 R_X86_64_GLOB_DAT 0000000000000000 __start_crt_compiler + 0

Relocation section '.rela.plt' at offset 0xb88 contains 5 entries:

Offset Info Type Sym. Value Sym. Name + Addend

000000500f40 000e00000007 R_X86_64_JUMP_SLO 0000000000400c10 exit + 0
000000500f48 001000000007 R_X86_64_JUMP_SLO 0000000000400c20 _exit + 0
000000500f60 000600000007 R_X86_64_JUMP_SLO 0000000000400c30 atexit + 0
000000500f70 001300000007 R_X86_64_JUMP_SLO 0000000000000000 __start_crt_compiler + 0
000000500f80 000700000007 R_X86_64_JUMP_SLO 0000000000400c50 __fpstart + 0


% readelf -r y64

Relocation section '.SUNW_reloc' at offset 0xaf8 contains 5 entries:

Offset Info Type Sym. Value Sym. Name + Addend

000000500fa8 000100000006 R_X86_64_GLOB_DAT 0000000000000000 _Jv_RegisterClasses + 0
000000500f98 000300000006 R_X86_64_GLOB_DAT 0000000000000000 _ITM_deregisterTMClone + 0
000000500fa0 000800000006 R_X86_64_GLOB_DAT 0000000000000000 _ITM_registerTMCloneTa + 0
000000500fb0 000d00000006 R_X86_64_GLOB_DAT 0000000000000000 fun + 0
000000500f78 001300000006 R_X86_64_GLOB_DAT 0000000000000000 __start_crt_compiler + 0

Relocation section '.rela.plt' at offset 0xb70 contains 6 entries:

Offset Info Type Sym. Value Sym. Name + Addend

000000500f50 000e00000007 R_X86_64_JUMP_SLO 0000000000400c10 exit + 0
000000500f58 001000000007 R_X86_64_JUMP_SLO 0000000000400c20 _exit + 0
000000500f70 000600000007 R_X86_64_JUMP_SLO 0000000000400c30 atexit + 0
000000500f80 001300000007 R_X86_64_JUMP_SLO 0000000000000000 __start_crt_compiler + 0
000000500f90 000700000007 R_X86_64_JUMP_SLO 0000000000400c50 __fpstart + 0
000000500fb8 000d00000007 R_X86_64_JUMP_SLO 0000000000000000 fun + 0