[native-client-discuss] __tls_.* X PT_TLS

[Rafael Espindola]

In NaCl we use linker scripts to define some __tls variables to find
the location of every TLS variable in memory. The normal ELF way to do
it is to put those in a different program segment
(http://www.sco.com/developers/gabi/latest/ch5.pheader.html#tls).

Is there any reason why we decided to do it differently?

Cheers,
--
Rafael Ávila de Espíndola

--
You received this message because you are subscribed to the Google Groups "Native-Client-Discuss" group.
To post to this group, send email to native-cli...@googlegroups.com.
To unsubscribe from this group, send email to native-client-di...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/native-client-discuss?hl=en.

[Mark Seaborn]

On Sat, May 15, 2010 at 8:04 PM, Rafael Espindola <espi...@google.com> wrote:

In NaCl we use linker scripts to define some __tls variables to find
the location of every TLS variable in memory. The normal ELF way to do
it is to put those in a different program segment
(http://www.sco.com/developers/gabi/latest/ch5.pheader.html#tls).

Is there any reason why we decided to do it differently?

Yes.  Normally (e.g. under Linux), libc will iterate through the executable's ELF Program Headers at startup to find the PT_TLS entry, which works because the Program Headers are mapped into memory at the start of the executable's text segment (the code + read-only data segment). 

Under NaCl, we can't include the ELF Program Headers in the code segment because they won't pass the validator.  So a statically-linked NaCl executable must find its TLS templates via statically-resolved symbols rather than reading from the PHDRs at run time.

For dynamically-linked NaCl executables and libraries, the dynamic linker will get TLS info from PHDRs, but the PHDRs still can't be included in the code segment.

Mark

[Rafael Espindola]

> Yes.  Normally (e.g. under Linux), libc will iterate through the
> executable's ELF Program Headers at startup to find the PT_TLS entry, which
> works because the Program Headers are mapped into memory at the start of the
> executable's text segment (the code + read-only data segment).
>
> Under NaCl, we can't include the ELF Program Headers in the code segment
> because they won't pass the validator.  So a statically-linked NaCl
> executable must find its TLS templates via statically-resolved symbols
> rather than reading from the PHDRs at run time.

Ah, that explains the comment on the linker script about the program
headers not being in a loaded segment. Thanks!

The early stages of the loader is a trusted code base, no? Can't it
read the information from the file and pass it on the side to the
runtime? If that is to cumbersome, I will probably send a patch
upstream adding the definitions of __tls_* symbols. It looks like a
generally useful thing to have and not too NaCl specific.

> For dynamically-linked NaCl executables and libraries, the dynamic linker
> will get TLS info from PHDRs, but the PHDRs still can't be included in the
> code segment.
>
> Mark
>

Thanks!

--
Rafael Ávila de Espíndola

[Mark Seaborn]

On Sun, May 16, 2010 at 4:38 PM, Rafael Espindola <espi...@google.com> wrote:

> Yes.  Normally (e.g. under Linux), libc will iterate through the
> executable's ELF Program Headers at startup to find the PT_TLS entry, which
> works because the Program Headers are mapped into memory at the start of the
> executable's text segment (the code + read-only data segment).
>
> Under NaCl, we can't include the ELF Program Headers in the code segment
> because they won't pass the validator.  So a statically-linked NaCl
> executable must find its TLS templates via statically-resolved symbols
> rather than reading from the PHDRs at run time.

Ah, that explains the comment on the linker script about the program
headers not being in a loaded segment. Thanks!

The early stages of the loader is a trusted code base, no?

Yes, loading the initial executable is done by sel_ldr.
 

Can't it read the information from the file and pass it on the side to the runtime?

Yes, it could, but what mechanism would you use to pass this information, and why would you want to do this?  The existing __tls_* symbols work well enough without adding logic to the TCB.

If that is to cumbersome, I will probably send a patch
upstream adding the definitions of __tls_* symbols. It looks like a
generally useful thing to have and not too NaCl specific.

Do you mean upstream to binutils?  Sounds OK, though I do wonder if it makes sense to upstream the binutils changes piecemeal.  We do require other linker script changes and this __tls_* change might not be very useful on its own.

Cheers,
Mark

[Rafael Espindola]

>> Can't it read the information from the file and pass it on the side to the
>> runtime?
>
> Yes, it could, but what mechanism would you use to pass this information,
> and why would you want to do this?  The existing __tls_* symbols work well
> enough without adding logic to the TCB.

The idea was to have more vanilla ELF files, but having less trusted
code is probably more important.

>> If that is to cumbersome, I will probably send a patch
>> upstream adding the definitions of __tls_* symbols. It looks like a
>> generally useful thing to have and not too NaCl specific.
>
> Do you mean upstream to binutils?  Sounds OK, though I do wonder if it makes
> sense to upstream the binutils changes piecemeal.  We do require other
> linker script changes and this __tls_* change might not be very useful on
> its own.

So, what we have that is different from the standard script is

*) Program header is not loaded
*) Segments are aligned to 64k
*) __tls_* symbols
*) Special initial addresses.

Is that it? If so I will try to send patches to gold in the background
to implement it. Gold does support scripts (I am using it for
testing), but is faster when not using one. The last item is in fact
already implemented as a command line option.

> Cheers,
> Mark

Cheers,

--
Rafael Ávila de Espíndola

[Mark Seaborn]

On Sun, May 16, 2010 at 11:25 PM, Rafael Espindola <espi...@google.com> wrote:

>> If that is to cumbersome, I will probably send a patch
>> upstream adding the definitions of __tls_* symbols. It looks like a
>> generally useful thing to have and not too NaCl specific.
>
> Do you mean upstream to binutils?  Sounds OK, though I do wonder if it makes
> sense to upstream the binutils changes piecemeal.  We do require other
> linker script changes and this __tls_* change might not be very useful on
> its own.

So, what we have that is different from the standard script is

*) Program header is not loaded
*) Segments are aligned to 64k
*) __tls_* symbols
*) Special initial addresses.

There are some more differences:

* Code and read-only data have to be split.  This involves
- a PHDRS declaration;
- moving all the .rel.* sections (for dynamic linking).

* We need to stick "=0x90909090" on the code sections in order to get NOP filling in the gaps.  Outside of NaCl, these aren't necessary, but the default script happens to have them for .init, .fini and .text.  For NaCl, these are necessary for all code sections.  (glibc uses some other section names.)  I've just raised http://code.google.com/p/nativeclient/issues/detail?id=499 which describes an alternative to putting this in the linker script.

Is that it? If so I will try to send patches to gold in the background
to implement it. Gold does support scripts (I am using it for
testing), but is faster when not using one.  The last item is in fact
already implemented as a command line option.

Interesting.  Is anyone linking code for NaCl yet that is big enough for Gold's speedups to be worthwhile?  Or are there customisations where "ld" has become too difficult to change?  ("ld" does have some odd behaviour in corner cases and last year I was wondering if gold would behave better.)

Cheers,
Mark

[Rafael Espindola]

> Interesting.  Is anyone linking code for NaCl yet that is big enough for
> Gold's speedups to be worthwhile?  Or are there customisations where "ld"
> has become too difficult to change?  ("ld" does have some odd behaviour in
> corner cases and last year I was wondering if gold would behave better.)

On the speed side I am not sure, but gold is interesting to PNaCl for
supporting plugins. Right now we use two linkers, gnu ld and llvm-ld
plus some very ugly hacks for them not agreeing about which object
files have to be included from the archives.

I also find command line option a bit more maintainable than linker
scripts. They are a bit more code to start with (c++), but become part
of the regular flow instead of a fully independent linker script.

I hope to have gold working (with linker scripts) today. Assuming that
works, I will open a low priority bug to port the linker script
functionality we need to gold (modulo things we are not sure should be
there, like the .palign issue you mentioned).

> Cheers,
> Mark
>

Cheers,
--
Rafael Ávila de Espíndola