History of semantic interposition & -fpic suppressing interprocedural optimizations
162 views
Skip to first unread message
Fangrui Song
May 8, 2021, 2:44:17 PM5/8/21
to Generic System V Application Binary Interface
Motivated by a recent Linux kernel rant on slow clang (libLLVM.so libclang-cpp.so) ( https://lore.kernel.org/lkml/CAHk-=whs8QZf3YnifdLv57+FhBi5_WeNTG1B-suOES=RcU...@mail.gmail.com/ )
and a post by Daniel Colascione (https://www.facebook.com/dan.colascione/posts/10107358290728348),
I am trying to push forward clang -fpic defaulting to -fno-semantic-interposition.
I am composing an article: https://maskray.me/blog/2021-05-09-fno-semantic-interposition
and want to add a bit more information about the history of ELF semantic interposition
and compilers' (does GCC inherit from its predecessors?) traditional -fpic behavior about suppressing interprocedural optimizations including inlining.
Anyone happy to share a bit on them? :)
H.J. Lu
May 8, 2021, 7:59:10 PM5/8/21
to Generic System V Application Binary Interface
of object files compiled by different compilers and with different
compiler options
at link time as well as shared libraries built by different compilers
and with different
compiler options at link time.
--
H.J.
H.J. Lu
May 8, 2021, 8:09:33 PM5/8/21
to Generic System V Application Binary Interface
--
H.J.
Fangrui Song
May 8, 2021, 9:37:19 PM5/8/21
to Generic System V Application Binary Interface
clang -fpic never suppresses interprocedural optimizations like gcc -fpic does.
clang non-x86 -fno-function-sections never supported function semantic interposition before 2020-01.
While I understand that it'd be difficult for GCC change (but individual distributions can introduce a configure-time option to test the default -fno-semantic-interposition behavior), the Clang change can be much smoother.
Since Clang 11, users who want the default GCC -fpic behavior can specify -fsemantic-interposition.
{Clang,GCC}x{ld.bfd,gold,ld.lld} cooperation works pretty well.
--
H.J.
Michael Matz
May 10, 2021, 11:28:20 AM5/10/21
to 'Fangrui Song' via Generic System V Application Binary Interface
Hello,
I'm not sure about the exact history. But I think the ELF behaviour is
the only strictly logical (if suboptimal for some cases). Observe that
this aspect of lookup rules only matters if there are multiple definitions
of the same symbol in a namespace.
* when only object files are involved: multiple defs are an error
* when archives are involved: first definition wins, the lookup chain
is _linear_, from left to right on the cmdline.
(static archives don't depend on other ones)
Now, think about what should happen when shared libraries are involved.
One basic premise of ELF is that they should behave as similar as possible
to static archives. So, for symbol lookup "the first one" should still
win. Of course, shared libraries have dependencies, so the definition of
"first one" needs a linearization of all involved components. Either a
global one for the whole image, or one per point of lookup; in absence of
any other consideration a single global linearization makes the most
sense. The obvious choices for linearizing the dependency graph would be
DFS and BFS. As 2nd-level dependencies are, well, 2nd-level, it makes
more sense to use BFS: after all, when linking with -la -lb and looking
for foo, when liba doesn't and libb does define foo, we expect to find the
one in libb, not one potentially defined in liba's dependencies.
With time the last item was even made more precise in that symbol lookup
at link editing time doesn't even look into 2nd level deps (depending on
linker flags and default behaviour, but still). But at runtime lookup the
existing rules couldn't be changed, BFS still needs to be used.
There's another invariant for ELF that follows from the above, but it
helps to spell it out: looking up a global symbol within a full process
image, finds one and only one definition, no matter from where that lookup
started (ignore details of lookup that can change this, like direct
binding). That is what makes it possible to talk about "the" definition
to start with. It's an invariant that's also given by the static archive
analogy. It should be noted that at least for non-functions this is not
just a theoretical excercise: there can be globally unique data that's
defined multiple times in order to ensure that it's always defined, but
that at runtime must all resolve to a single entity. Problems with
details about that lead to the introduction of STB_GNU_UNIQUE, but the
initial ELF rules already provided the necessary basic guarantees.
But the above rules have some implications, and one of them is the
possibility of interposition. Not just via pre-loading, but also e.g. by
updating liba to also contain a definition of foo (in the above -la -lb
example). It's just natural (though possibly unexpected) that now the foo
in liba is found globally. That has it's disadvantages of course. (And
it's advantages).
Now, compilers. Obviously it makes sense if conforming source code,
compiled for a specific target platform behaves the same, no matter which
compiler or compilation options are chosen. In particular if compiled
with or without optimizations should better not matter. I use conforming
in a broad sense: not just adhering to the various language standard (say
ISO C), but rather extended to expect features that are guaranteed by the
platform to work. E.g. ISO C would forbid multiple global definition of
the same symbol, and none of the above would matter the slightest. But in
real life those happen. If they happen, for ELF targets, the ELF spec
says how they should behave.
Given the constraint that -O0 and -O2 should behave the same, plus the
constraint how ELF symbol lookup behaves, it follows that inlining (and
similar interprocedural transformations) must be disabled for code that
could tell the difference, i.e. code that's intended to be placed in
shared libraries on ELF platforms.
So, a question about the clang behaviour (seemingly historic, now with a
flag, but still enabled by default): in presence of a program that can
tell the difference, is clang behaving different between -O0 and -O2? If
so, do you think that's acceptable?
Ciao,
Michael.
the only strictly logical (if suboptimal for some cases). Observe that
this aspect of lookup rules only matters if there are multiple definitions
of the same symbol in a namespace.
* when only object files are involved: multiple defs are an error
* when archives are involved: first definition wins, the lookup chain
is _linear_, from left to right on the cmdline.
(static archives don't depend on other ones)
Now, think about what should happen when shared libraries are involved.
One basic premise of ELF is that they should behave as similar as possible
to static archives. So, for symbol lookup "the first one" should still
win. Of course, shared libraries have dependencies, so the definition of
"first one" needs a linearization of all involved components. Either a
global one for the whole image, or one per point of lookup; in absence of
any other consideration a single global linearization makes the most
sense. The obvious choices for linearizing the dependency graph would be
DFS and BFS. As 2nd-level dependencies are, well, 2nd-level, it makes
more sense to use BFS: after all, when linking with -la -lb and looking
for foo, when liba doesn't and libb does define foo, we expect to find the
one in libb, not one potentially defined in liba's dependencies.
With time the last item was even made more precise in that symbol lookup
at link editing time doesn't even look into 2nd level deps (depending on
linker flags and default behaviour, but still). But at runtime lookup the
existing rules couldn't be changed, BFS still needs to be used.
There's another invariant for ELF that follows from the above, but it
helps to spell it out: looking up a global symbol within a full process
image, finds one and only one definition, no matter from where that lookup
started (ignore details of lookup that can change this, like direct
binding). That is what makes it possible to talk about "the" definition
to start with. It's an invariant that's also given by the static archive
analogy. It should be noted that at least for non-functions this is not
just a theoretical excercise: there can be globally unique data that's
defined multiple times in order to ensure that it's always defined, but
that at runtime must all resolve to a single entity. Problems with
details about that lead to the introduction of STB_GNU_UNIQUE, but the
initial ELF rules already provided the necessary basic guarantees.
But the above rules have some implications, and one of them is the
possibility of interposition. Not just via pre-loading, but also e.g. by
updating liba to also contain a definition of foo (in the above -la -lb
example). It's just natural (though possibly unexpected) that now the foo
in liba is found globally. That has it's disadvantages of course. (And
it's advantages).
Now, compilers. Obviously it makes sense if conforming source code,
compiled for a specific target platform behaves the same, no matter which
compiler or compilation options are chosen. In particular if compiled
with or without optimizations should better not matter. I use conforming
in a broad sense: not just adhering to the various language standard (say
ISO C), but rather extended to expect features that are guaranteed by the
platform to work. E.g. ISO C would forbid multiple global definition of
the same symbol, and none of the above would matter the slightest. But in
real life those happen. If they happen, for ELF targets, the ELF spec
says how they should behave.
Given the constraint that -O0 and -O2 should behave the same, plus the
constraint how ELF symbol lookup behaves, it follows that inlining (and
similar interprocedural transformations) must be disabled for code that
could tell the difference, i.e. code that's intended to be placed in
shared libraries on ELF platforms.
So, a question about the clang behaviour (seemingly historic, now with a
flag, but still enabled by default): in presence of a program that can
tell the difference, is clang behaving different between -O0 and -O2? If
so, do you think that's acceptable?
Ciao,
Michael.
Fangrui Song
May 11, 2021, 6:22:49 PM5/11/21
to gener...@googlegroups.com
On 2021-05-10, Michael Matz wrote:
>Hello,
Thanks for your reply.
I know there is a DSO-archive analogy but wonder whether it serves as a good
analogy. An object file shadows an archive member if the archive member is not
extracted. If the archive member is extracted (it essentially becomes an object
file), there will still be a multiple definition error.
The error case is different from the result we get by linking together a.o and
b.so.
>Now, think about what should happen when shared libraries are involved.
>One basic premise of ELF is that they should behave as similar as possible
>to static archives. So, for symbol lookup "the first one" should still
>win. Of course, shared libraries have dependencies, so the definition of
>"first one" needs a linearization of all involved components. Either a
>global one for the whole image, or one per point of lookup; in absence of
>any other consideration a single global linearization makes the most
>sense.
I am on the fence about the choice. But I understand that ELF uses breadth-first
search and POSIX requires dlsym with an explicit handle to use breadth-first
search (rooted at the argument, not the global symbol resolution order).
>The obvious choices for linearizing the dependency graph would be
>DFS and BFS. As 2nd-level dependencies are, well, 2nd-level, it makes
>more sense to use BFS: after all, when linking with -la -lb and looking
>for foo, when liba doesn't and libb does define foo, we expect to find the
>one in libb, not one potentially defined in liba's dependencies.
Makes sense. I'll add a note that POSIX says (for dlopen) during relocation
processing load ordering (DFS) is used.
>With time the last item was even made more precise in that symbol lookup
>at link editing time doesn't even look into 2nd level deps (depending on
>linker flags and default behaviour, but still). But at runtime lookup the
>existing rules couldn't be changed, BFS still needs to be used.
This reminds me of --no-allow-shlib-undefined. The gold and LLD's implementation
just inspects the direct dependencies. This turns out to be a better quality
assurance feature: link what you use like include what you use. The fully
specified direct dependencies make library organizing in a large system more
robust.
>There's another invariant for ELF that follows from the above, but it
>helps to spell it out: looking up a global symbol within a full process
>image, finds one and only one definition, no matter from where that lookup
>started (ignore details of lookup that can change this, like direct
>binding). That is what makes it possible to talk about "the" definition
>to start with. It's an invariant that's also given by the static archive
>analogy. It should be noted that at least for non-functions this is not
>just a theoretical excercise: there can be globally unique data that's
>defined multiple times in order to ensure that it's always defined, but
>that at runtime must all resolve to a single entity. Problems with
>details about that lead to the introduction of STB_GNU_UNIQUE, but the
>initial ELF rules already provided the necessary basic guarantees.
(Do we find problems where STB_GNU_UNIQUE fails? I know that it implies
DF_1_NODELETE which may make some folks unhappy)
>But the above rules have some implications, and one of them is the
>possibility of interposition. Not just via pre-loading, but also e.g. by
>updating liba to also contain a definition of foo (in the above -la -lb
>example). It's just natural (though possibly unexpected) that now the foo
>in liba is found globally. That has it's disadvantages of course. (And
>it's advantages).
Folks from some other camps may find this behavior surprising.
I understand that there are both advantages and disadvantages.
>Now, compilers. Obviously it makes sense if conforming source code,
>compiled for a specific target platform behaves the same, no matter which
>compiler or compilation options are chosen. In particular if compiled
>with or without optimizations should better not matter. I use conforming
>in a broad sense: not just adhering to the various language standard (say
>ISO C), but rather extended to expect features that are guaranteed by the
>platform to work. E.g. ISO C would forbid multiple global definition of
>the same symbol, and none of the above would matter the slightest. But in
>real life those happen. If they happen, for ELF targets, the ELF spec
>says how they should behave.
>
>Given the constraint that -O0 and -O2 should behave the same, plus the
>constraint how ELF symbol lookup behaves, it follows that inlining (and
>similar interprocedural transformations) must be disabled for code that
>could tell the difference, i.e. code that's intended to be placed in
>shared libraries on ELF platforms.
>
>So, a question about the clang behaviour (seemingly historic, now with a
>flag, but still enabled by default): in presence of a program that can
>tell the difference, is clang behaving different between -O0 and -O2? If
>so, do you think that's acceptable?
inline int f(int x) { ... }
int g(int x) { return f(x); }
In gcc -fpic mode, -O0 may not inline f while -O2 may inline -O2.
-fno-semantic-interposition extends this distinction to non-inline functions
(external linkage in LLVM IR).
If we switch to a -fno-semantic-interposition by default world,
we can make -O0 and -O2 consistent: -O0 will call f with a non-preemptible localalias
while -O2 may inline f or apply interprocedural optimizations.
In both cases interposition on f may not hijack g's callee.
>
>Ciao,
>Michael.
>
>--
>You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.
>To view this discussion on the web visit https://groups.google.com/d/msgid/generic-abi/alpine.LSU.2.20.2105101444130.18262%40wotan.suse.de.
>Hello,
Thanks for your reply.
analogy. An object file shadows an archive member if the archive member is not
extracted. If the archive member is extracted (it essentially becomes an object
file), there will still be a multiple definition error.
The error case is different from the result we get by linking together a.o and
b.so.
>Now, think about what should happen when shared libraries are involved.
>One basic premise of ELF is that they should behave as similar as possible
>to static archives. So, for symbol lookup "the first one" should still
>win. Of course, shared libraries have dependencies, so the definition of
>"first one" needs a linearization of all involved components. Either a
>global one for the whole image, or one per point of lookup; in absence of
>any other consideration a single global linearization makes the most
>sense.
search and POSIX requires dlsym with an explicit handle to use breadth-first
search (rooted at the argument, not the global symbol resolution order).
>The obvious choices for linearizing the dependency graph would be
>DFS and BFS. As 2nd-level dependencies are, well, 2nd-level, it makes
>more sense to use BFS: after all, when linking with -la -lb and looking
>for foo, when liba doesn't and libb does define foo, we expect to find the
>one in libb, not one potentially defined in liba's dependencies.
processing load ordering (DFS) is used.
>With time the last item was even made more precise in that symbol lookup
>at link editing time doesn't even look into 2nd level deps (depending on
>linker flags and default behaviour, but still). But at runtime lookup the
>existing rules couldn't be changed, BFS still needs to be used.
just inspects the direct dependencies. This turns out to be a better quality
assurance feature: link what you use like include what you use. The fully
specified direct dependencies make library organizing in a large system more
robust.
>There's another invariant for ELF that follows from the above, but it
>helps to spell it out: looking up a global symbol within a full process
>image, finds one and only one definition, no matter from where that lookup
>started (ignore details of lookup that can change this, like direct
>binding). That is what makes it possible to talk about "the" definition
>to start with. It's an invariant that's also given by the static archive
>analogy. It should be noted that at least for non-functions this is not
>just a theoretical excercise: there can be globally unique data that's
>defined multiple times in order to ensure that it's always defined, but
>that at runtime must all resolve to a single entity. Problems with
>details about that lead to the introduction of STB_GNU_UNIQUE, but the
>initial ELF rules already provided the necessary basic guarantees.
DF_1_NODELETE which may make some folks unhappy)
>But the above rules have some implications, and one of them is the
>possibility of interposition. Not just via pre-loading, but also e.g. by
>updating liba to also contain a definition of foo (in the above -la -lb
>example). It's just natural (though possibly unexpected) that now the foo
>in liba is found globally. That has it's disadvantages of course. (And
>it's advantages).
I understand that there are both advantages and disadvantages.
>Now, compilers. Obviously it makes sense if conforming source code,
>compiled for a specific target platform behaves the same, no matter which
>compiler or compilation options are chosen. In particular if compiled
>with or without optimizations should better not matter. I use conforming
>in a broad sense: not just adhering to the various language standard (say
>ISO C), but rather extended to expect features that are guaranteed by the
>platform to work. E.g. ISO C would forbid multiple global definition of
>the same symbol, and none of the above would matter the slightest. But in
>real life those happen. If they happen, for ELF targets, the ELF spec
>says how they should behave.
>
>Given the constraint that -O0 and -O2 should behave the same, plus the
>constraint how ELF symbol lookup behaves, it follows that inlining (and
>similar interprocedural transformations) must be disabled for code that
>could tell the difference, i.e. code that's intended to be placed in
>shared libraries on ELF platforms.
>
>So, a question about the clang behaviour (seemingly historic, now with a
>flag, but still enabled by default): in presence of a program that can
>tell the difference, is clang behaving different between -O0 and -O2? If
>so, do you think that's acceptable?
int g(int x) { return f(x); }
In gcc -fpic mode, -O0 may not inline f while -O2 may inline -O2.
-fno-semantic-interposition extends this distinction to non-inline functions
(external linkage in LLVM IR).
If we switch to a -fno-semantic-interposition by default world,
we can make -O0 and -O2 consistent: -O0 will call f with a non-preemptible localalias
while -O2 may inline f or apply interprocedural optimizations.
In both cases interposition on f may not hijack g's callee.
>
>Ciao,
>Michael.
>
>--
>You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.
>To view this discussion on the web visit https://groups.google.com/d/msgid/generic-abi/alpine.LSU.2.20.2105101444130.18262%40wotan.suse.de.
Florian Weimer
Jun 9, 2021, 3:00:04 AM6/9/21
to 'Fangrui Song' via Generic System V Application Binary Interface
* via Generic System V. Application Binary Interface:
> I am trying to push forward clang -fpic defaulting to
> -fno-semantic-interposition.
My hunch is that on several targets, you need more than that to get a
clearly observable effect (beyond startup time); basically -flto with a
version script for the linker and a linker LTO plugin that reports
per-symbol visibility information back to the compiler. x86-64 (with
the small code model) is very good at relaxing relocations, but that's
not feasible everywhere. One might still encounter unnecessary
indirections even with -fno-semantic-interposition -Bsymbolic.
That seems to go beyond the scope of the ELF specification.
Thanks,
Florian
> I am trying to push forward clang -fpic defaulting to
> -fno-semantic-interposition.
clearly observable effect (beyond startup time); basically -flto with a
version script for the linker and a linker LTO plugin that reports
per-symbol visibility information back to the compiler. x86-64 (with
the small code model) is very good at relaxing relocations, but that's
not feasible everywhere. One might still encounter unnecessary
indirections even with -fno-semantic-interposition -Bsymbolic.
That seems to go beyond the scope of the ELF specification.
Thanks,
Florian
Fāng-ruì Sòng
Jun 9, 2021, 3:19:18 AM6/9/21
to Generic System V Application Binary Interface
interposition is still possible (i.e. taking the address of a function
does not use the local alias).
Semantic interposition is disabled (i.e. interposition cannot change
the semantics).
It seems that **not using** a local alias for variable access is
intended by GCC -fno-semantic-interposition.
If variable access/function address uses a local alias as well,
-fno-semantic-interposition -shared -Bsymbolic would not have worse
code generation than -fpie -pie.
As of why protected symbols cannot be used, because protected symbols
have been made very broken in GCC/binutils, some time circa 2016, more
so on x86.
If someone wants to know more details,
https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected
-fvisibility=protected is too coarse-grained. An option not applying
to weak symbols might be more useful.
Due to copy relocations, a variant not applying to variables would be
useful as well.
Florian Weimer
Jun 9, 2021, 3:41:08 AM6/9/21
to 'Fāng-ruì Sòng' via Generic System V Application Binary Interface
* 'Fāng-ruì Sòng' via Generic System V Application Binary Interface:
> If variable access/function address uses a local alias as well,
> -fno-semantic-interposition -shared -Bsymbolic would not have worse
> code generation than -fpie -pie.
Yes, but for C/C++, generating that local alias needs LTO because you
can't see from the declarations in a header file whether something is
local to the current ELF shared object/executable or not.
It's possible to add the required aliases manually without LTO, but the
results can be quite complex. We do it in glibc, and we are hitting
limits around IFUNCs (can't use STV_HIDDEN for them on x86-64, for
example), and it generally doesn't scale from a maintenance perspective.
The macro goo for that keeps spreading. For data access in glibc, I
want to switch to hand-crafted lazy resolution for most symbols because
pretty much all programs only ever access around 10% of the global data
symbols in glibc.
> -fvisibility=protected is too coarse-grained. An option not applying
> to weak symbols might be more useful.
> Due to copy relocations, a variant not applying to variables would be
> useful as well.
H.J. Lu is working on phasing out copy relocations on x86-64. With LTO,
I do not think they are needed anymore. The tricky aspect is how to do
this without impacting backwards compatibility.
Thanks,
Florian
> If variable access/function address uses a local alias as well,
> -fno-semantic-interposition -shared -Bsymbolic would not have worse
> code generation than -fpie -pie.
can't see from the declarations in a header file whether something is
local to the current ELF shared object/executable or not.
It's possible to add the required aliases manually without LTO, but the
results can be quite complex. We do it in glibc, and we are hitting
limits around IFUNCs (can't use STV_HIDDEN for them on x86-64, for
example), and it generally doesn't scale from a maintenance perspective.
The macro goo for that keeps spreading. For data access in glibc, I
want to switch to hand-crafted lazy resolution for most symbols because
pretty much all programs only ever access around 10% of the global data
symbols in glibc.
> -fvisibility=protected is too coarse-grained. An option not applying
> to weak symbols might be more useful.
> Due to copy relocations, a variant not applying to variables would be
> useful as well.
I do not think they are needed anymore. The tricky aspect is how to do
this without impacting backwards compatibility.
Thanks,
Florian
Fāng-ruì Sòng
Jun 9, 2021, 4:16:40 AM6/9/21
to Generic System V Application Binary Interface
On Wed, Jun 9, 2021 at 12:41 AM Florian Weimer <fwe...@redhat.com> wrote:
>
> * 'Fāng-ruì Sòng' via Generic System V Application Binary Interface:
>
> > If variable access/function address uses a local alias as well,
> > -fno-semantic-interposition -shared -Bsymbolic would not have worse
> > code generation than -fpie -pie.
>
> Yes, but for C/C++, generating that local alias needs LTO because you
> can't see from the declarations in a header file whether something is
> local to the current ELF shared object/executable or not.
For
>
> * 'Fāng-ruì Sòng' via Generic System V Application Binary Interface:
>
> > If variable access/function address uses a local alias as well,
> > -fno-semantic-interposition -shared -Bsymbolic would not have worse
> > code generation than -fpie -pie.
>
> Yes, but for C/C++, generating that local alias needs LTO because you
> can't see from the declarations in a header file whether something is
> local to the current ELF shared object/executable or not.
extern int var;
int foo() { return var; }
yes, LTO is needed to avoid the GOT indirection.
The Windows approach is like assigning undefined symbols a protected visibility.
Then all undefined symbols don't need GOT indirection.
The downside is that every undefined symbol which may be defined in an
external component will need explicit annotation,
which is against the ELF spirit
https://maskray.me/blog/2021-05-16-elf-interposition-and-bsymbolic#no-source-level-annotation
> It's possible to add the required aliases manually without LTO, but the
> results can be quite complex. We do it in glibc, and we are hitting
> limits around IFUNCs (can't use STV_HIDDEN for them on x86-64, for
> example),
efficiently emulated by a global function pointer.
It is something like -fno-plt - expanding the PLT call sequence in the
call site.
> and it generally doesn't scale from a maintenance perspective.
> The macro goo for that keeps spreading. For data access in glibc, I
> want to switch to hand-crafted lazy resolution for most symbols because
> pretty much all programs only ever access around 10% of the global data
> symbols in glibc.
of any properly written application.
And if it actually becomes a bottleneck I don't mind that the compiler
cannot optimize it more :/
https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected#summary
lists a number of function related issues.
I care more about the function issues more and don't think any GNU
PROPERTY is needed to fix the GCC/binutils issues.
Actually clang + gold/ld.lld has done quite a few things correctly:)
> > -fvisibility=protected is too coarse-grained. An option not applying
> > to weak symbols might be more useful.
> > Due to copy relocations, a variant not applying to variables would be
> > useful as well.
>
> H.J. Lu is working on phasing out copy relocations on x86-64. With LTO,
> I do not think they are needed anymore. The tricky aspect is how to do
> this without impacting backwards compatibility.
without LTO.
That is the idea behind ld -Bsemantic-non-weak-functions (I hope it
can become distribution default) +
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100593#c9
> Thanks,
> Florian
>
> --
> You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/generic-abi/8735tr8oz5.fsf%40oldenburg.str.redhat.com.
> --
> You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.
Michael Matz
Jun 9, 2021, 9:48:08 AM6/9/21
to 'Fāng-ruì Sòng' via Generic System V Application Binary Interface
Hello,
On Wed, 9 Jun 2021, 'Fāng-ruì Sòng' via Generic System V Application
Binary Interface wrote:
> I was confused by GCC's -fno-semantic-interposition. It means address
> interposition is still possible (i.e. taking the address of a function
> does not use the local alias).
> Semantic interposition is disabled (i.e. interposition cannot change
> the semantics).
>
> It seems that **not using** a local alias for variable access is
> intended by GCC -fno-semantic-interposition.
>
> If variable access/function address uses a local alias as well,
> -fno-semantic-interposition -shared -Bsymbolic would not have worse
> code generation than -fpie -pie.
And would break existing binaries containing copy relocs.
Btw: I have a hunch that you are concentrating too much on a single
solution to your initial problem (as you said it is was a complaint from a
user about clang's slowness). Your solution to that is to completely
change the symbol interposition rules of ELF. Are you sure it's the only
way? E.g. _what_ is slow without the change? Is it symbol lookup itself,
or the impossibility to inline functions because of interposition
requirements?
Because for both I can envision different solutions that aren't turning
interposition off. Because I like interposition, in fact I think it's a
very good feature of ELF.
But e.g. interposition could be limited to symbols where it traditionally
has it's uses: C symbols. It could for instance be switched off for C++
class methods (at which point the vtable references could then also become
non-symbolic). It could also be switched off for functions resulting from
template instantiation.
If you were to propose that, and where also proposing to make that default
for a distro I would be in your camp probably. But I would not consider
making anything switching off interposition globally the default for our
distro.
(GNU ld has collected bits of pieces of such approaches over time, e.g.
-Bsymbolic-functions and --dynamic-list-{data,cpp-new,cpp-typeinfo}, but
none of them really match what would be ideal)
> As of why protected symbols cannot be used, because protected symbols
> have been made very broken in GCC/binutils, some time circa 2016, more
> so on x86. If someone wants to know more details,
> https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected
Yes, copy relocations should be phased out, no disagreement here.
Canonical PLT entries: as well for protected functions, for default ones:
hmm.
But we must deal with the problem of existing uses of these undesirable
features.
> -fvisibility=protected is too coarse-grained. An option not applying to
> weak symbols might be more useful. Due to copy relocations, a variant
> not applying to variables would be useful as well.
I once hacked something for GCC that made all functions protected, i.e.
-fvisbility-functions=protected, in order to get the inlining benefit from
the associated non-preemptability. Then I realized that -fpie gave me
essentially the same semantics (for what I wanted) :)
Ciao,
Michael.
On Wed, 9 Jun 2021, 'Fāng-ruì Sòng' via Generic System V Application
Binary Interface wrote:
> I was confused by GCC's -fno-semantic-interposition. It means address
> interposition is still possible (i.e. taking the address of a function
> does not use the local alias).
> Semantic interposition is disabled (i.e. interposition cannot change
> the semantics).
>
> It seems that **not using** a local alias for variable access is
> intended by GCC -fno-semantic-interposition.
>
> If variable access/function address uses a local alias as well,
> -fno-semantic-interposition -shared -Bsymbolic would not have worse
> code generation than -fpie -pie.
Btw: I have a hunch that you are concentrating too much on a single
solution to your initial problem (as you said it is was a complaint from a
user about clang's slowness). Your solution to that is to completely
change the symbol interposition rules of ELF. Are you sure it's the only
way? E.g. _what_ is slow without the change? Is it symbol lookup itself,
or the impossibility to inline functions because of interposition
requirements?
Because for both I can envision different solutions that aren't turning
interposition off. Because I like interposition, in fact I think it's a
very good feature of ELF.
But e.g. interposition could be limited to symbols where it traditionally
has it's uses: C symbols. It could for instance be switched off for C++
class methods (at which point the vtable references could then also become
non-symbolic). It could also be switched off for functions resulting from
template instantiation.
If you were to propose that, and where also proposing to make that default
for a distro I would be in your camp probably. But I would not consider
making anything switching off interposition globally the default for our
distro.
(GNU ld has collected bits of pieces of such approaches over time, e.g.
-Bsymbolic-functions and --dynamic-list-{data,cpp-new,cpp-typeinfo}, but
none of them really match what would be ideal)
> As of why protected symbols cannot be used, because protected symbols
> have been made very broken in GCC/binutils, some time circa 2016, more
> so on x86. If someone wants to know more details,
> https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected
Canonical PLT entries: as well for protected functions, for default ones:
hmm.
But we must deal with the problem of existing uses of these undesirable
features.
> -fvisibility=protected is too coarse-grained. An option not applying to
> weak symbols might be more useful. Due to copy relocations, a variant
> not applying to variables would be useful as well.
-fvisbility-functions=protected, in order to get the inlining benefit from
the associated non-preemptability. Then I realized that -fpie gave me
essentially the same semantics (for what I wanted) :)
Ciao,
Michael.
Fāng-ruì Sòng
Jun 9, 2021, 12:47:01 PM6/9/21
to Generic System V Application Binary Interface
On Wed, Jun 9, 2021 at 6:48 AM Michael Matz <ma...@suse.de> wrote:
>
> Hello,
Hello:)
> On Wed, 9 Jun 2021, 'Fāng-ruì Sòng' via Generic System V Application
> Binary Interface wrote:
>
> > I was confused by GCC's -fno-semantic-interposition. It means address
> > interposition is still possible (i.e. taking the address of a function
> > does not use the local alias).
> > Semantic interposition is disabled (i.e. interposition cannot change
> > the semantics).
> >
> > It seems that **not using** a local alias for variable access is
> > intended by GCC -fno-semantic-interposition.
> >
> > If variable access/function address uses a local alias as well,
> > -fno-semantic-interposition -shared -Bsymbolic would not have worse
> > code generation than -fpie -pie.
>
> And would break existing binaries containing copy relocs.
>
> Btw: I have a hunch that you are concentrating too much on a single
> solution to your initial problem (as you said it is was a complaint from a
> user about clang's slowness). Your solution to that is to completely
> change the symbol interposition rules of ELF. Are you sure it's the only
> way? E.g. _what_ is slow without the change? Is it symbol lookup itself,
> or the impossibility to inline functions because of interposition
> requirements?
The sheer number of symbol lookups R_*_JUMP_SLOT/R_X86_64_64 and a few
R_*_GLOB_DAT was the **main reason** behind
the Fedora clang's slowness (and Linus' rant).
I rebuilt my dynamically linked Clang with -Wl,-Bsymbolic-functions
(80+% fewer symbol lookups) and my Linux kernel x86-64 defconfig build
with clang is more than 15% faster.
The reason is likely because there are too many small files and the
ld.so time takes a significant portion of a short-lived process.
Reducing the number of dynamic relocations (either fixing
STV_PROTECTED or using a variant of -Bsymbolic-functions) is the only
direction for such applications.
-fno-semantic-interposition contributed an extra 3% boost on some
benchmarks, but not for my Linux kernel build.
> Because for both I can envision different solutions that aren't turning
> interposition off. Because I like interposition, in fact I think it's a
> very good feature of ELF.
>
> But e.g. interposition could be limited to symbols where it traditionally
> has it's uses: C symbols. It could for instance be switched off for C++
> class methods (at which point the vtable references could then also become
> non-symbolic). It could also be switched off for functions resulting from
> template instantiation.
For C++, vague-linkage definitions (inline functions, template
instantiations, etc) don't suppress inlining in GCC.
inline void foo() {}
void bar() { foo() }
Interposition on foo does not guarantee to work.
IIUC GCC -fno-semantic-interposition applies to non-vague-linkage
external linkage definitions.
Hmmm, making -fno-semantic-interposition default for all C++ symbols
(C++ has the one definition rule) would be useful.
Doing that for C will be a plus. For example, there was a large cost
for CPython (27%).
I know some users do LD_PRELOAD. But more often the LD_PRELOAD usage
applies to an undefined symbol
(https://maskray.me/blog/2021-05-16-elf-interposition-and-bsymbolic#interaction-with-ld_preload).
It is very rare that a user replaces a STB_GLOBAL STV_DEFAULT
function. Such tricks, when rarely applied, are to work around some
bad software.
For such cases the software should just be re-built without aggressive
symbolic link options.
On Mach-O, some tricks like -flat_namespace are needed to allow interposition.
IIUC Solaris does allow interposition to override direct bindings.
> If you were to propose that, and where also proposing to make that default
> for a distro I would be in your camp probably. But I would not consider
> making anything switching off interposition globally the default for our
> distro.
>
> (GNU ld has collected bits of pieces of such approaches over time, e.g.
> -Bsymbolic-functions and --dynamic-list-{data,cpp-new,cpp-typeinfo}, but
> none of them really match what would be ideal)
Yep:) I'd even say --dynamic-list-{data,cpp-new,cpp-typeinfo} has zero
use in the wild.
> > As of why protected symbols cannot be used, because protected symbols
> > have been made very broken in GCC/binutils, some time circa 2016, more
> > so on x86. If someone wants to know more details,
>
> > https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected
>
> Yes, copy relocations should be phased out, no disagreement here.
> Canonical PLT entries: as well for protected functions, for default ones:
> hmm.
>
> But we must deal with the problem of existing uses of these undesirable
> features.
>
> > -fvisibility=protected is too coarse-grained. An option not applying to
> > weak symbols might be more useful. Due to copy relocations, a variant
> > not applying to variables would be useful as well.
>
> I once hacked something for GCC that made all functions protected, i.e.
> -fvisbility-functions=protected, in order to get the inlining benefit from
> the associated non-preemptability. Then I realized that -fpie gave me
> essentially the same semantics (for what I wanted) :)
My idea is that -fvisibility=protected -fpic is very similar to -fpie,
but it cannot be used due to copy relocations and canonical PLT
entries.
The slightly inferior -fno-semantic-interposition -fpic is also
similar to -fpie, just not optimization global variables (which should
not matter for performance anyway).
-fno-semantic-interposition works well with
-fvisibility-inlindes-hidden, when users don't want pointer equality
for functions. As my article says, such property is rarely leveraged.
It is difficult to guarantee on Windows anyway.
However, -fpie also optimizes TLS, which makes it not suitable for
-shared linking.
I think the -fvisibility variant should exclude variables and exclude
weak definitions.
If you still insist on C symbols interposable by default, I think you
may make that -f option g++/CXXFLAGS only...
If you change your mind in the future, you can apply that -f option to
gcc/CFLAGS as well..
Such an option will be mostly suitable as the distribution default.
https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected#protected-function-symbols-and-canonical-plt-entries
The GNU ld aarch64 and x86's rejection on this code should be fixed:
__attribute__((visibility("protected"))) void *foo () {
return (void *)foo;
}
% gcc -fpic -shared b.c -fuse-ld=bfd b.c -o b.so
/usr/bin/ld.bfd: /tmp/cc3Ay0Gh.o: relocation R_X86_64_PC32 against
protected symbol `foo' can not be used when making a shared object
/usr/bin/ld.bfd: final link failed: bad value
collect2: error: ld returned 1 exit status
% gcc -shared -fuse-ld=bfd -fpic b.c -o b.so
/usr/bin/ld.bfd: /tmp/ccXdBqMf.o: relocation
R_AARCH64_ADR_PREL_PG_HI21 against symbol `foo' which may bind
externally can not be used when making a shared object; recompile with
-fPIC
/tmp/ccXdBqMf.o: in function `foo':
a.c:(.text+0x0): dangerous relocation: unsupported relocation
collect2: error: ld returned 1 exit status
The diagnostic was to make a fragile glibc feature (acked by multiple
glibc developers): copy relocation on protected symbols.
Unfortunately it applies to function symbols as well.
>
> Ciao,
> Michael.
>
> Hello,
Hello:)
> On Wed, 9 Jun 2021, 'Fāng-ruì Sòng' via Generic System V Application
> Binary Interface wrote:
>
> > I was confused by GCC's -fno-semantic-interposition. It means address
> > interposition is still possible (i.e. taking the address of a function
> > does not use the local alias).
> > Semantic interposition is disabled (i.e. interposition cannot change
> > the semantics).
> >
> > It seems that **not using** a local alias for variable access is
> > intended by GCC -fno-semantic-interposition.
> >
> > If variable access/function address uses a local alias as well,
> > -fno-semantic-interposition -shared -Bsymbolic would not have worse
> > code generation than -fpie -pie.
>
> And would break existing binaries containing copy relocs.
>
> Btw: I have a hunch that you are concentrating too much on a single
> solution to your initial problem (as you said it is was a complaint from a
> user about clang's slowness). Your solution to that is to completely
> change the symbol interposition rules of ELF. Are you sure it's the only
> way? E.g. _what_ is slow without the change? Is it symbol lookup itself,
> or the impossibility to inline functions because of interposition
> requirements?
R_*_GLOB_DAT was the **main reason** behind
the Fedora clang's slowness (and Linus' rant).
I rebuilt my dynamically linked Clang with -Wl,-Bsymbolic-functions
(80+% fewer symbol lookups) and my Linux kernel x86-64 defconfig build
with clang is more than 15% faster.
The reason is likely because there are too many small files and the
ld.so time takes a significant portion of a short-lived process.
Reducing the number of dynamic relocations (either fixing
STV_PROTECTED or using a variant of -Bsymbolic-functions) is the only
direction for such applications.
-fno-semantic-interposition contributed an extra 3% boost on some
benchmarks, but not for my Linux kernel build.
> Because for both I can envision different solutions that aren't turning
> interposition off. Because I like interposition, in fact I think it's a
> very good feature of ELF.
>
> But e.g. interposition could be limited to symbols where it traditionally
> has it's uses: C symbols. It could for instance be switched off for C++
> class methods (at which point the vtable references could then also become
> non-symbolic). It could also be switched off for functions resulting from
> template instantiation.
instantiations, etc) don't suppress inlining in GCC.
inline void foo() {}
void bar() { foo() }
Interposition on foo does not guarantee to work.
IIUC GCC -fno-semantic-interposition applies to non-vague-linkage
external linkage definitions.
Hmmm, making -fno-semantic-interposition default for all C++ symbols
(C++ has the one definition rule) would be useful.
Doing that for C will be a plus. For example, there was a large cost
for CPython (27%).
I know some users do LD_PRELOAD. But more often the LD_PRELOAD usage
applies to an undefined symbol
(https://maskray.me/blog/2021-05-16-elf-interposition-and-bsymbolic#interaction-with-ld_preload).
It is very rare that a user replaces a STB_GLOBAL STV_DEFAULT
function. Such tricks, when rarely applied, are to work around some
bad software.
For such cases the software should just be re-built without aggressive
symbolic link options.
On Mach-O, some tricks like -flat_namespace are needed to allow interposition.
IIUC Solaris does allow interposition to override direct bindings.
> If you were to propose that, and where also proposing to make that default
> for a distro I would be in your camp probably. But I would not consider
> making anything switching off interposition globally the default for our
> distro.
>
> (GNU ld has collected bits of pieces of such approaches over time, e.g.
> -Bsymbolic-functions and --dynamic-list-{data,cpp-new,cpp-typeinfo}, but
> none of them really match what would be ideal)
use in the wild.
> > As of why protected symbols cannot be used, because protected symbols
> > have been made very broken in GCC/binutils, some time circa 2016, more
> > so on x86. If someone wants to know more details,
>
> > https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected
>
> Yes, copy relocations should be phased out, no disagreement here.
> Canonical PLT entries: as well for protected functions, for default ones:
> hmm.
>
> But we must deal with the problem of existing uses of these undesirable
> features.
>
> > -fvisibility=protected is too coarse-grained. An option not applying to
> > weak symbols might be more useful. Due to copy relocations, a variant
> > not applying to variables would be useful as well.
>
> I once hacked something for GCC that made all functions protected, i.e.
> -fvisbility-functions=protected, in order to get the inlining benefit from
> the associated non-preemptability. Then I realized that -fpie gave me
> essentially the same semantics (for what I wanted) :)
but it cannot be used due to copy relocations and canonical PLT
entries.
The slightly inferior -fno-semantic-interposition -fpic is also
similar to -fpie, just not optimization global variables (which should
not matter for performance anyway).
-fno-semantic-interposition works well with
-fvisibility-inlindes-hidden, when users don't want pointer equality
for functions. As my article says, such property is rarely leveraged.
It is difficult to guarantee on Windows anyway.
However, -fpie also optimizes TLS, which makes it not suitable for
-shared linking.
I think the -fvisibility variant should exclude variables and exclude
weak definitions.
If you still insist on C symbols interposable by default, I think you
may make that -f option g++/CXXFLAGS only...
If you change your mind in the future, you can apply that -f option to
gcc/CFLAGS as well..
Such an option will be mostly suitable as the distribution default.
https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected#protected-function-symbols-and-canonical-plt-entries
The GNU ld aarch64 and x86's rejection on this code should be fixed:
__attribute__((visibility("protected"))) void *foo () {
return (void *)foo;
}
% gcc -fpic -shared b.c -fuse-ld=bfd b.c -o b.so
/usr/bin/ld.bfd: /tmp/cc3Ay0Gh.o: relocation R_X86_64_PC32 against
protected symbol `foo' can not be used when making a shared object
/usr/bin/ld.bfd: final link failed: bad value
collect2: error: ld returned 1 exit status
% gcc -shared -fuse-ld=bfd -fpic b.c -o b.so
/usr/bin/ld.bfd: /tmp/ccXdBqMf.o: relocation
R_AARCH64_ADR_PREL_PG_HI21 against symbol `foo' which may bind
externally can not be used when making a shared object; recompile with
-fPIC
/tmp/ccXdBqMf.o: in function `foo':
a.c:(.text+0x0): dangerous relocation: unsupported relocation
collect2: error: ld returned 1 exit status
The diagnostic was to make a fragile glibc feature (acked by multiple
glibc developers): copy relocation on protected symbols.
Unfortunately it applies to function symbols as well.
>
> Ciao,
> Michael.
>
> --
> You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/generic-abi/alpine.LSU.2.22.394.2106091309100.3803%40wotan.suse.de.
> --
> You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages