STT_NOTYPE Specification

[Suprateeka R Hegde]

Hi

I see that on Linux, an undefined function reference (with a proper
prototype) gets STT_NOTYPE. Does gABI mandate anything in this regard?
The one liner specification of STT_NOTYPE does not seem to say anything
about associating it with functions. Instead STT_FUNC is a close match.

For instance,

$ cat notype.c
extern void foo(void);
int main(){ foo(); return 0; }

And then readelf(1) shows:
8: 0000000000000000 21 FUNC GLOBAL DEFAULT 1 main
9: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND foo

And then, assume there is another file with a definition of foo as
integer (STT_OBJECT). Say, int foo = 99; linking this with the notype.c
works absoultely fine with no warnings/diagnsotics. Of course when run
it gets a SEGV.

Tried the same on other platforms like HP-UX and Solaris with gcc(1), I
see that the undefined function reference gets STT_FUNC as type and
eventually the linker warns on type mismatch.

So what does gABI specify for this case?

--
Supra

[Suprateeka R Hegde]

(Adding to my earlier mail)

The x86-64 ABI says:
---
... if an executable file references a function defined in a shared
object, the link editor will place the address of the procedure linkage
table entry for that function in its associated symbol table entry. This
will result in symbol table entries with section index of SHN_UNDEF but
a type of STT_FUNC and a non-zero st_value
---

This behavior is seen on other platforms I mentioned earlier, but not on
Linux x86-64 (with no special options to gcc).

--
Supra

[H.J. Lu]

On Thu, Jul 2, 2015 at 10:45 AM, Suprateeka R Hegde
<hegdes...@gmail.com> wrote:
> (Adding to my earlier mail)
>
> The x86-64 ABI says:
> ---
> ... if an executable file references a function defined in a shared object,
> the link editor will place the address of the procedure linkage table entry
> for that function in its associated symbol table entry. This will result in
> symbol table entries with section index of SHN_UNDEF but a type of STT_FUNC
> and a non-zero st_value
> ---
>
> This behavior is seen on other platforms I mentioned earlier, but not on
> Linux x86-64 (with no special options to gcc).

See

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=35514


--
H.J.

[Cary Coutant]

>> I see that on Linux, an undefined function reference (with a proper
>> prototype) gets STT_NOTYPE. Does gABI mandate anything in this regard?
>> The one liner specification of STT_NOTYPE does not seem to say anything
>> about associating it with functions. Instead STT_FUNC is a close match.
>>
>> For instance,
>>
>> $ cat notype.c
>> extern void foo(void);
>> int main(){ foo(); return 0; }
>>
>> And then readelf(1) shows:
>> 8: 0000000000000000 21 FUNC GLOBAL DEFAULT 1 main
>> 9: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND foo
>>
>> And then, assume there is another file with a definition of foo as
>> integer (STT_OBJECT). Say, int foo = 99; linking this with the notype.c
>> works absoultely fine with no warnings/diagnsotics. Of course when run
>> it gets a SEGV.
>>
>> Tried the same on other platforms like HP-UX and Solaris with gcc(1), I
>> see that the undefined function reference gets STT_FUNC as type and
>> eventually the linker warns on type mismatch.
>>
>> So what does gABI specify for this case?

The gABI doesn't require anything; it simply says that STT_NOTYPE
means that the type is unspecified.

> (Adding to my earlier mail)
>
> The x86-64 ABI says:
> ---
> ... if an executable file references a function defined in a shared object,
> the link editor will place the address of the procedure linkage table entry
> for that function in its associated symbol table entry. This will result in
> symbol table entries with section index of SHN_UNDEF but a type of STT_FUNC
> and a non-zero st_value
> ---
>
> This behavior is seen on other platforms I mentioned earlier, but not on
> Linux x86-64 (with no special options to gcc).

This is talking about executables, and the result of the link, not
relocatables. If you have an undef STT_NOTYPE that binds to an
STT_FUNC in a shared library, gold sets the undef to STT_FUNC, while
Gnu ld leaves it as STT_NOTYPE.

-cary

[H.J. Lu]

On Thu, Jul 2, 2015 at 11:27 AM, Cary Coutant <ccou...@gmail.com> wrote:

>> This behavior is seen on other platforms I mentioned earlier, but not on
>> Linux x86-64 (with no special options to gcc).
>
> This is talking about executables, and the result of the link, not
> relocatables. If you have an undef STT_NOTYPE that binds to an
> STT_FUNC in a shared library, gold sets the undef to STT_FUNC, while
> Gnu ld leaves it as STT_NOTYPE.
>

That is not what I see:

[hjl@gnu-6 type-2]$ make
gcc -O -g -B./ -c -o x.o x.c
gcc -O -g -B./ -o x x.o
readelf -sW x.o | grep puts
15: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND puts
readelf -sW x | grep puts
1: 0000000000000000 0 FUNC GLOBAL DEFAULT UND puts@GLIBC_2.2.5 (2)
54: 0000000000000000 0 FUNC GLOBAL DEFAULT UND puts@@GLIBC_2.2.5
[hjl@gnu-6 type-2]$

puts is NOTYPE in x.o and FUNC in x.


--
H.J.

[Suprateeka R Hegde]

On Thursday 02 July 2015 11:57 PM, Cary Coutant wrote:
>>> I see that on Linux, an undefined function reference (with a proper
>>> prototype) gets STT_NOTYPE. Does gABI mandate anything in this regard?
>>> The one liner specification of STT_NOTYPE does not seem to say anything
>>> about associating it with functions. Instead STT_FUNC is a close match.
>>>
>>> For instance,
>>>
>>> $ cat notype.c
>>> extern void foo(void);
>>> int main(){ foo(); return 0; }
>>>
>>> And then readelf(1) shows:
>>> 8: 0000000000000000 21 FUNC GLOBAL DEFAULT 1 main
>>> 9: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND foo
>>>
>>> And then, assume there is another file with a definition of foo as
>>> integer (STT_OBJECT). Say, int foo = 99; linking this with the notype.c
>>> works absoultely fine with no warnings/diagnsotics. Of course when run
>>> it gets a SEGV.
>>>
>>> Tried the same on other platforms like HP-UX and Solaris with gcc(1), I
>>> see that the undefined function reference gets STT_FUNC as type and
>>> eventually the linker warns on type mismatch.
>>>
>>> So what does gABI specify for this case?
>
> The gABI doesn't require anything; it simply says that STT_NOTYPE
> means that the type is unspecified.

So I assume that it is incorrect for gcc(1) to mark an undefined
function reference as STT_NOTYPE. I think that is the bug HJ pointed out.

>> (Adding to my earlier mail)
>>
>> The x86-64 ABI says:
>> ---
>> ... if an executable file references a function defined in a shared object,
>> the link editor will place the address of the procedure linkage table entry
>> for that function in its associated symbol table entry. This will result in
>> symbol table entries with section index of SHN_UNDEF but a type of STT_FUNC
>> and a non-zero st_value
>> ---
>>
>> This behavior is seen on other platforms I mentioned earlier, but not on
>> Linux x86-64 (with no special options to gcc).
>
> This is talking about executables, and the result of the link, not
> relocatables.

I understand. It is not directly related to the case I explained. I
mentioned that because I found differences between GNU ld and gold.

> If you have an undef STT_NOTYPE that binds to an
> STT_FUNC in a shared library, gold sets the undef to STT_FUNC, while
> Gnu ld leaves it as STT_NOTYPE.

Basically gold turns STT_NOTYPE to the type of first definition it finds
on the link line. If it finds an STT_OBJECT of the same name as the
undefined function, then the function symbol is turned to STT_OBJECT
type (with SHN_UNDEF in case of -fpic) in the executable.

There is definitely a need for the compiler to emit the type so that
linkers can warn on type mismatch. I recently stumbled upon a huge code
base and the link went through fine. As I mentioned it resulted in a
SEGV at runtime. I agree it is badly written code, but then that is why
we have warnings and diagnostics.

I have another thought. What is wrong in mandating through standards
that when the compiler knows a symbol is of type STT_FUNC, it should not
mark it as STT_NOTYPE. Is there any compatibility issue? Does it break
any conventions? I could not think of any.

--
Supra

[Cary Coutant]

> That is not what I see:
>
> [hjl@gnu-6 type-2]$ make
> gcc -O -g -B./ -c -o x.o x.c
> gcc -O -g -B./ -o x x.o
> readelf -sW x.o | grep puts
> 15: 0000000000000000 0 NOTYPE GLOBAL DEFAULT UND puts
> readelf -sW x | grep puts
> 1: 0000000000000000 0 FUNC GLOBAL DEFAULT UND puts@GLIBC_2.2.5 (2)
> 54: 0000000000000000 0 FUNC GLOBAL DEFAULT UND puts@@GLIBC_2.2.5
> [hjl@gnu-6 type-2]$
>
> puts is NOTYPE in x.o and FUNC in x.

My quick test, using a source file I had lying around, had a weak
unsat to foo(). If I take out the weak attribute, then Gnu ld also
sets foo to STT_FUNC in the a.out.

-cary

[Cary Coutant]

>> The gABI doesn't require anything; it simply says that STT_NOTYPE
>> means that the type is unspecified.
>
> So I assume that it is incorrect for gcc(1) to mark an undefined function
> reference as STT_NOTYPE. I think that is the bug HJ pointed out.

No, I don't think it's incorrect at all. It's merely a
quality-of-implementation issue: setting the undef to STT_FUNC or
STT_OBJECT allows the linker to diagnose a mismatch between reference
and definition, but, traditionally, Unix linkers have always been
happy to bind symbols without checking types, and I'll bet there's
still plenty of code that depends on that.

-cary

[Suprateeka R Hegde]

On Friday 03 July 2015 12:32 AM, Suprateeka R Hegde wrote:
> I found differences between GNU ld and gold

That seems to be some mistake I did. Looks like I have compared with
some other linkers. GNU ld and gold both are same in this regard.

On Friday 03 July 2015 01:59 AM, Cary Coutant wrote:
>>> The gABI doesn't require anything; it simply says that STT_NOTYPE
>>> means that the type is unspecified.
>>
>> So I assume that it is incorrect for gcc(1) to mark an undefined function
>> reference as STT_NOTYPE. I think that is the bug HJ pointed out.
>
> No, I don't think it's incorrect at all. It's merely a
> quality-of-implementation issue:

I agree, but there is a very thin line of differnece between standards
and quality. I would say standards drive the quality.

With so many additions to standards, may it be gABI, psABI, ISO/IEC
C/C++ or anything else, the quality of developer tool chain (that
complies to standards) are great today, compared to what it was
traditionally.

> setting the undef to STT_FUNC or
> STT_OBJECT allows the linker to diagnose a mismatch between reference
> and definition,

Yes, and it indeed helps a lot to know that a function is being bound to
an integer (which makes no sense). Its just like compilers warn/error if
it sees a declaration and definition mismatch in the same translation unit.

BTW, I binary edited and changed STT_NOTYPE of that undefined func to
STT_FUNC. I didnt know what function attribute in gcc(1) would help me
get a STT_FUNC with SHN_UNDEF. Even with that binary change, neither
ld.bfd nor ld.gold warned/diagnosed there is a mismatch. They
successfully bound the STT_FUNC/SHN_UNDEF with STT_OBJECT (from the
shared object) and in the resulting executable, the STT_FUNC/SHN_UNDEF
is changed to STT_OBJECT. And of course SEGV at rutime.

However, I got the expected warning/diagnostic with linkers on HP-UX and
Solaris (SPARC).

On the lighter side of it, I feel compilers are waiting for linkers to
honour type mismatch and linkers are waiting for compilers to emit type
info :-)

> but, traditionally, Unix linkers have always been
> happy to bind symbols without checking types, and I'll bet there's
> still plenty of code that depends on that.

I absolutely agree on the existing code depending on this behavior. But
then we (as compiler-n-linker/runtime developers) do need to improve the
tool-chain to be in-sync with the programming trend.

If I want to claim, yes, XandY are all state-of-the-art compiler and
linker tool chain, then I should ensure that:
- A new engineer working on a huge piece of undocumented legacy code,
should not find a SEGV at runtime, just because an undefined
"is_something_true()", binds to his newly inserted boolean
"is_something_true". The tool chain should warn/diagnose in that case
and aid in the develolment.
- An application developer need not worry too much on the system side of
it. Instead write code for readability and maintainability.
- An application developer should not worry too much if his code affects
for instance, loop-unrolling badly or prevents a
static-to-global-promotion, affects register allocation, etc. Its should
be our responsibility to detect them and advise/warn/diagnose.

I absolutelty agree that standards are not supposed to mandate all these
and they are just quality-of-implementation issues. But then standards
do help in a lot of way to make excellent tool chain. There are
instances where a simple port of code base from KnR C to C99 standards
have removed lots of potential bugs, as the new standard dis-allows
certain ambiguous constructs.

I was actually thinking, just like language standards, why not gABI also
have something like SYSV Generic ABI 98, 99, 2011, 2014, etc? And let
the newer versions be stricter. Let the tool chain implement options
accordingly (something like -gabi=99, -gabi=14, etc). This would not
break legacy code and compatibility.

I also want to know why should symbol resolution and symbol types be
tied to processors? I see that AArch64 ELF ABI says, "any symbol can be
of type STT_NOTYPE...". The AMD64 ABI does not say anything like that. I
agree that symbol handling is coupled with relocations, but something
generic and basic like "do not bind a function to integer" need not be
processor specific. Do I miss anything here?

Or is there any platform where the calling conventions allow binding a
function reference to user defined data (not plt/got/etc) with the fact
that data can hold address of a function?

--
Supra