RFC: Add R_X86_64_GPOFF/R_386_GPOFF relocation

[H.J. Lu]

On x86, a segment register is used as thread pointer to access thread
local storage (TLS) with __thread. TLS is an OS-dependent, user-space
feature.

Here is a proposal to add R_X86_64_GPOFF/R_386_GPOFF relocation to
access symbol with an offset to global pointer, __gp. The assembly
syntax is

1. Load value of foo relative to __gp, pointed by %seg, into %reg:

op %seg:foo@GPOFF, %reg

2. Store value in %reg to foo relative to __gp, pointed by %seg:

op %reg, %seg:foo@GPOFF

3. Compute offset of foo to __gp:

lea foo@GPOFF, %reg
.long foo@GPOFF

Linker sets __gp to the middle of the section which contains definitions
of symbols with GPOFF relocations and the maximum offset is [-2G, 2G).
Run-time must initialize the segment register, %seg, with the address of
global pointer, __gp.

Any comments?

--
H.J.

[Florian Weimer]

Is this related to GCC's address space support?

Shouldn't the relocation encode which segment register to use? In
general, this information cannot be recovered from the instruction stream.

Note that %gs (or %fs on i386) is currently essentially a callee-saved
global register variable, and you cannot repurpose for something else
without breaking some applications.

Florian

[H.J. Lu]

On Mon, Aug 7, 2017 at 4:08 AM, Florian Weimer <fwe...@redhat.com> wrote:
> On 08/06/2017 07:03 PM, H.J. Lu wrote:
>> On x86, a segment register is used as thread pointer to access thread
>> local storage (TLS) with __thread. TLS is an OS-dependent, user-space
>> feature.
>>
>> Here is a proposal to add R_X86_64_GPOFF/R_386_GPOFF relocation to
>> access symbol with an offset to global pointer, __gp. The assembly
>> syntax is
>>
>> 1. Load value of foo relative to __gp, pointed by %seg, into %reg:
>>
>> op %seg:foo@GPOFF, %reg
>>
>> 2. Store value in %reg to foo relative to __gp, pointed by %seg:
>>
>> op %reg, %seg:foo@GPOFF
>>
>> 3. Compute offset of foo to __gp:
>>
>> lea foo@GPOFF, %reg
>> .long foo@GPOFF
>>
>> Linker sets __gp to the middle of the section which contains definitions
>> of symbols with GPOFF relocations and the maximum offset is [-2G, 2G).
>> Run-time must initialize the segment register, %seg, with the address of
>> global pointer, __gp.
>>
>> Any comments?
>
> Is this related to GCC's address space support?

Yes.

> Shouldn't the relocation encode which segment register to use? In
> general, this information cannot be recovered from the instruction stream.

This information isn't needed by linker and there is no %seg in

lea foo@GPOFF, %reg

which computes the offset from __gp.

> Note that %gs (or %fs on i386) is currently essentially a callee-saved
> global register variable, and you cannot repurpose for something else
> without breaking some applications.

That is true. Only the unused segment register can be used. Here is
the binutils prototype at

https://sourceware.org/git/?p=binutils-gdb.git;a=shortlog;h=refs/heads/users/hjl/gpoff

with 2 run-time tests each for Linux/i386 and Linux/x86-64.


--
H.J.

[Florian Weimer]

On 08/07/2017 02:28 PM, H.J. Lu wrote:

>>> Linker sets __gp to the middle of the section which contains definitions
>>> of symbols with GPOFF relocations and the maximum offset is [-2G, 2G).
>>> Run-time must initialize the segment register, %seg, with the address of
>>> global pointer, __gp.
>>>
>>> Any comments?
>>
>> Is this related to GCC's address space support?
>
> Yes.

It probably needs debugger support and DWARF extensions, too.

>> Shouldn't the relocation encode which segment register to use? In
>> general, this information cannot be recovered from the instruction stream.
>
> This information isn't needed by linker and there is no %seg in
>
> lea foo@GPOFF, %reg
>
> which computes the offset from __gp.

Well, the offset is still specific to the segment register being. But
if there continue to be just two of them with flexible base offsets, and
one of them is reserved for the implementation of TLS, then it might be
unambiguous.

>> Note that %gs (or %fs on i386) is currently essentially a callee-saved
>> global register variable, and you cannot repurpose for something else
>> without breaking some applications.
>
> That is true. Only the unused segment register can be used.

There is no unused segment register I know of. %gs is used by
applications. The base offset is thread-local in Wine, so this support
doesn't really help there, I think, because the __gp seems to be global.

Florian

[H.J. Lu]

%gs isn't used by all applications on x86-64.

> doesn't really help there, I think, because the __gp seems to be global.
>

__gp is a hidden global symbol and unique to each ELF module.



--
H.J.

[Florian Weimer]

On 08/07/2017 02:53 PM, H.J. Lu wrote:
>> There is no unused segment register I know of. %gs is used by
>> applications. The base offset is thread-local in Wine, so this support
> %gs isn't used by all applications on x86-64.

But it's used by some. If we start to use it in the rest of the system,
those applications will break. If we start using it inside glibc, it is
possible that Wine will never work again.

>> doesn't really help there, I think, because the __gp seems to be global.
>>
> __gp is a hidden global symbol and unique to each ELF module.

Right, so it's not useful for TLS storage.

Florian

[H.J. Lu]

On Mon, Aug 7, 2017 at 5:56 AM, Florian Weimer <fwe...@redhat.com> wrote:
> On 08/07/2017 02:53 PM, H.J. Lu wrote:
>>> There is no unused segment register I know of. %gs is used by
>>> applications. The base offset is thread-local in Wine, so this support
>> %gs isn't used by all applications on x86-64.
>
> But it's used by some. If we start to use it in the rest of the system,
> those applications will break. If we start using it inside glibc, it is
> possible that Wine will never work again.

It should be only used by applications, not in glibc.

>>> doesn't really help there, I think, because the __gp seems to be global.
>>>
>> __gp is a hidden global symbol and unique to each ELF module.
>
> Right, so it's not useful for TLS storage.
>

That is true.


--
H.J.

[Florian Weimer]

On 08/07/2017 03:11 PM, H.J. Lu wrote:
> On Mon, Aug 7, 2017 at 5:56 AM, Florian Weimer <fwe...@redhat.com> wrote:
>> On 08/07/2017 02:53 PM, H.J. Lu wrote:
>>>> There is no unused segment register I know of. %gs is used by
>>>> applications. The base offset is thread-local in Wine, so this support
>>> %gs isn't used by all applications on x86-64.
>>
>> But it's used by some. If we start to use it in the rest of the system,
>> those applications will break. If we start using it inside glibc, it is
>> possible that Wine will never work again.
>
> It should be only used by applications, not in glibc.

But if X, Mesa and their dependencies (which includes LLVM on some
systems) start using it, Wine will still be broken.

Thanks,
Florian

[H.J. Lu]

On Mon, Aug 7, 2017 at 6:05 AM, Michael Matz <ma...@suse.de> wrote:
> Hi,

>
> On Mon, 7 Aug 2017, H.J. Lu wrote:
>
>> >>>> Linker sets __gp to the middle of the section which contains definitions
>> >>>> of symbols with GPOFF relocations and the maximum offset is [-2G, 2G).
>> >>>> Run-time must initialize the segment register, %seg, with the address of
>> >>>> global pointer, __gp.
>> >>>>
>> >>>> Any comments?
>> >>>
>> >>> Is this related to GCC's address space support?
>> >>
>> >> Yes.
>

> Before removing the last segment register from general use (and hence
> break some existing apps) I'd want to see a much better rationale then
> "yeah, something with address spaces".

>
>> > doesn't really help there, I think, because the __gp seems to be global.
>>
>> __gp is a hidden global symbol and unique to each ELF module.
>

> E.g. if __gp is hidden and unique per ELF module (and hence not global at
> all!) why would .got not be enough?
>

[hjl@gnu-tools-1 tmp]$ cat x.c
int __seg_gs i;

int
foo (void)
{
return i;
}
[hjl@gnu-tools-1 tmp]$ gcc -S -O2 x.c
[hjl@gnu-tools-1 tmp]$ cat x.s
.file "x.c"
.text
.p2align 4,,15
.globl foo
.type foo, @function
foo:
.LFB0:
.cfi_startproc
movl %gs:i(%rip), %eax
ret
.cfi_endproc
.LFE0:
.size foo, .-foo
.globl i
.bss
.align 4
.type i, @object
.size i, 4
i:
.zero 4
.ident "GCC: (GNU) 7.1.1 20170709 (Red Hat 7.1.1-4)"
.section .note.GNU-stack,"",@progbits
[hjl@gnu-tools-1 tmp]$

GOT isn't used and GOT may be desirable in some cases.

--
H.J.

[H.J. Lu]

Are they using

int __seg_gs i;

or addressspace in llvm? If yes, how does Wine work?


--
H.J.

[H.J. Lu]

On Mon, Aug 7, 2017 at 7:18 AM, Michael Matz <ma...@suse.de> wrote:
> Hi,
>
> On Mon, 7 Aug 2017, H.J. Lu wrote:
>
>> > Before removing the last segment register from general use (and hence
>> > break some existing apps) I'd want to see a much better rationale then
>> > "yeah, something with address spaces".
>> >
>> >> > doesn't really help there, I think, because the __gp seems to be global.
>> >>
>> >> __gp is a hidden global symbol and unique to each ELF module.
>> >
>> > E.g. if __gp is hidden and unique per ELF module (and hence not global at
>> > all!) why would .got not be enough?
>> >
>>
>> [hjl@gnu-tools-1 tmp]$ cat x.c
>> int __seg_gs i;
>> int foo (void) { return i; }

> ...
>> movl %gs:i(%rip), %eax
> ...

>> GOT isn't used and GOT may be desirable in some cases.
>

> I don't see how this clears anything up or gives a rationale. If __gp is
> hidden local per ELF module (and hence _different_ per ELF module), then
> per your initial description:

>
> "Linker sets __gp to the middle of the section which contains
> definitions of symbols with GPOFF relocations and the maximum offset is
> [-2G, 2G). Run-time must initialize the segment register, %seg, with the
> address of global pointer, __gp."
>

> the above reference to 'i' (using the new proposed reloc, being an offset
> from %gs:0, which is specific to the ELF module containing that
> instruction) is implicitely hidden and local to the ELF module as well.
> At which point this all seems a bit silly as you could just as well also
> use a proper hidden symbol in local .bss/.data, addressed via PC-relative,
> or via GOT offset.
>
> Another problem with the above is that to have __gp (and therefore gs:0)
> be local and different per ELF module you'd have to actively change and
> restore it whenever you cross ELF module borders.
>
> If, OTOH you just mispecified the above and intended to leave __gp be
> indeed global and the same for all ELF modules, then you need the same
> mechanisms that currently deal with TLS storage also for this "global"
> segment (namely you need to construct one from all constituent ELF
> modules, and need to provision for the offsets into that constructed block
> to not be link time constants but only load time constants (when ignoring
> dlopen!) and so need some space to store these offsets in memory so that
> the loader can modify them).
>
> So, again, I'm still confused about your rationale and how you intend for
> all this to work together.
>

My proposal is the counter part of __seg_fs/__seg_gs in compiler. It
has its limitation and isn't a replacement of TLS. The same questions
can be asked for _seg_fs/__seg_gs.


--
H.J.

[Michael Matz]

Hi,

On Mon, 7 Aug 2017, H.J. Lu wrote:

> >>>> Linker sets __gp to the middle of the section which contains definitions
> >>>> of symbols with GPOFF relocations and the maximum offset is [-2G, 2G).
> >>>> Run-time must initialize the segment register, %seg, with the address of
> >>>> global pointer, __gp.
> >>>>
> >>>> Any comments?
> >>>
> >>> Is this related to GCC's address space support?
> >>
> >> Yes.

Before removing the last segment register from general use (and hence
break some existing apps) I'd want to see a much better rationale then
"yeah, something with address spaces".

> > doesn't really help there, I think, because the __gp seems to be global.
>
> __gp is a hidden global symbol and unique to each ELF module.

E.g. if __gp is hidden and unique per ELF module (and hence not global at
all!) why would .got not be enough?

Ciao,
Michael.

[Michael Matz]

Hi,

On Mon, 7 Aug 2017, H.J. Lu wrote:

> > Before removing the last segment register from general use (and hence
> > break some existing apps) I'd want to see a much better rationale then
> > "yeah, something with address spaces".
> >
> >> > doesn't really help there, I think, because the __gp seems to be global.
> >>
> >> __gp is a hidden global symbol and unique to each ELF module.
> >
> > E.g. if __gp is hidden and unique per ELF module (and hence not global at
> > all!) why would .got not be enough?
> >
>
> [hjl@gnu-tools-1 tmp]$ cat x.c
> int __seg_gs i;
> int foo (void) { return i; }

...
> movl %gs:i(%rip), %eax
...

> GOT isn't used and GOT may be desirable in some cases.

I don't see how this clears anything up or gives a rationale. If __gp is
hidden local per ELF module (and hence _different_ per ELF module), then
per your initial description:

"Linker sets __gp to the middle of the section which contains
definitions of symbols with GPOFF relocations and the maximum offset is
[-2G, 2G). Run-time must initialize the segment register, %seg, with the
address of global pointer, __gp."

the above reference to 'i' (using the new proposed reloc, being an offset
from %gs:0, which is specific to the ELF module containing that
instruction) is implicitely hidden and local to the ELF module as well.
At which point this all seems a bit silly as you could just as well also
use a proper hidden symbol in local .bss/.data, addressed via PC-relative,
or via GOT offset.

Another problem with the above is that to have __gp (and therefore gs:0)
be local and different per ELF module you'd have to actively change and
restore it whenever you cross ELF module borders.

If, OTOH you just mispecified the above and intended to leave __gp be
indeed global and the same for all ELF modules, then you need the same
mechanisms that currently deal with TLS storage also for this "global"
segment (namely you need to construct one from all constituent ELF
modules, and need to provision for the offsets into that constructed block
to not be link time constants but only load time constants (when ignoring
dlopen!) and so need some space to store these offsets in memory so that
the loader can modify them).

So, again, I'm still confused about your rationale and how you intend for
all this to work together.

Ciao,
Michael.