[llvm-dev] Targeting old glibc
Alexandre Bique via llvm-dev
I wonder what is the right way to target an old glibc?
I have a machine which is up to date (glibc 2.32 and clang+lld 10.0.1).
So far I've been able to target older glibc by having a C file containing:
__asm__(".symver powf,powf@GLIBC_2.2.5");
__asm__(".symver expf,expf@GLIBC_2.2.5");
__asm__(".symver exp2f,exp2f@GLIBC_2.2.5");
__asm__(".symver log2f,log2f@GLIBC_2.2.5");
__asm__(".symver logf,logf@GLIBC_2.2.5");
__asm__(".symver log,log@GLIBC_2.2.5");
__asm__(".symver log2,log2@GLIBC_2.2.5");
__asm__(".symver exp,exp@GLIBC_2.2.5");
__asm__(".symver exp2,exp2@GLIBC_2.2.5");
__asm__(".symver pow,pow@GLIBC_2.2.5");
But after updating clang and re-creating the cmake build directory it
does not work anymore and I have dependencies toward glibc 2.29:
nm -D /home/abique/.u-he/Zebra2/Zebra2.64.so | grep GLIBC.*29
U exp@@GLIBC_2.29
U exp2@@GLIBC_2.29
U log@@GLIBC_2.29
U pow@@GLIBC_2.29
At first I thought that it was due to LTO, but even after disabling
LTO the problem still occurs.
How to solve it?
Is there a better approach to this problem?
Maybe lld never supported it, and now it is using lld to link instead
of the GNU linker?
Anyway, I'd like to solve this with the LLVM tools while full LTO is
enabled, which requires lld right?
Many thanks.
Regards,
Alexandre Bique
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Fangrui Song via llvm-dev
>Hi,
>
>I wonder what is the right way to target an old glibc?
>
>I have a machine which is up to date (glibc 2.32 and clang+lld 10.0.1).
>
>So far I've been able to target older glibc by having a C file containing:
>
>__asm__(".symver powf,powf@GLIBC_2.2.5");
>__asm__(".symver expf,expf@GLIBC_2.2.5");
>__asm__(".symver exp2f,exp2f@GLIBC_2.2.5");
>__asm__(".symver log2f,log2f@GLIBC_2.2.5");
>__asm__(".symver logf,logf@GLIBC_2.2.5");
>
>__asm__(".symver log,log@GLIBC_2.2.5");
>__asm__(".symver log2,log2@GLIBC_2.2.5");
>__asm__(".symver exp,exp@GLIBC_2.2.5");
>__asm__(".symver exp2,exp2@GLIBC_2.2.5");
>__asm__(".symver pow,pow@GLIBC_2.2.5");
This works. This approach is used by https://github.com/wheybags/glibc_version_header
>But after updating clang and re-creating the cmake build directory it
>does not work anymore and I have dependencies toward glibc 2.29:
>
>nm -D /home/abique/.u-he/Zebra2/Zebra2.64.so | grep GLIBC.*29
> U exp@@GLIBC_2.29
> U exp2@@GLIBC_2.29
> U log@@GLIBC_2.29
> U pow@@GLIBC_2.29
I reported a display problem. top-of-trunk binutils will display
undefined versioned symbols as @ instead of @@.
>At first I thought that it was due to LTO, but even after disabling
>LTO the problem still occurs.
>
>How to solve it?
>Is there a better approach to this problem?
>
>Maybe lld never supported it, and now it is using lld to link instead
>of the GNU linker?
LLD has good support for version symbols (you need a commit after https://reviews.llvm.org/D80059 (in 11.0.0, but not in 10.x)).
There is insufficient information to form a solution.
You may add -Wl,-y,exp,-y,exp@GLIBC_2.29,-y,exp@GLIBC_2.2.5 to figure
out why exp@GLIBC_2.29 instead of exp@GLIBC_2.2.5 is picked.
Alexandre Bique via llvm-dev
>
> On 2020-10-28, Alexandre Bique via llvm-dev wrote:
> >Hi,
> >
> >I wonder what is the right way to target an old glibc?
> >
> >I have a machine which is up to date (glibc 2.32 and clang+lld 10.0.1).
> >
> >So far I've been able to target older glibc by having a C file containing:
> >
> >__asm__(".symver powf,powf@GLIBC_2.2.5");
> >__asm__(".symver expf,expf@GLIBC_2.2.5");
> >__asm__(".symver exp2f,exp2f@GLIBC_2.2.5");
> >__asm__(".symver log2f,log2f@GLIBC_2.2.5");
> >__asm__(".symver logf,logf@GLIBC_2.2.5");
> >
> >__asm__(".symver log,log@GLIBC_2.2.5");
> >__asm__(".symver log2,log2@GLIBC_2.2.5");
> >__asm__(".symver exp,exp@GLIBC_2.2.5");
> >__asm__(".symver exp2,exp2@GLIBC_2.2.5");
> >__asm__(".symver pow,pow@GLIBC_2.2.5");
>
> This works. This approach is used by https://github.com/wheybags/glibc_version_header
Thank you, that is something useful!
> >But after updating clang and re-creating the cmake build directory it
> >does not work anymore and I have dependencies toward glibc 2.29:
> >
> >nm -D /home/abique/.u-he/Zebra2/Zebra2.64.so | grep GLIBC.*29
> > U exp@@GLIBC_2.29
> > U exp2@@GLIBC_2.29
> > U log@@GLIBC_2.29
> > U pow@@GLIBC_2.29
>
> I reported a display problem. top-of-trunk binutils will display
> undefined versioned symbols as @ instead of @@.
Thank you, I wondered why there were two @.
> >At first I thought that it was due to LTO, but even after disabling
> >LTO the problem still occurs.
> >
> >How to solve it?
> >Is there a better approach to this problem?
> >
> >Maybe lld never supported it, and now it is using lld to link instead
> >of the GNU linker?
>
> LLD has good support for version symbols (you need a commit after https://reviews.llvm.org/D80059 (in 11.0.0, but not in 10.x)).
I'm using Archlinux, and llvm 11 is in staging, I'll see if I can install it.
> There is insufficient information to form a solution.
> You may add -Wl,-y,exp,-y,exp@GLIBC_2.29,-y,exp@GLIBC_2.2.5 to figure
> out why exp@GLIBC_2.29 instead of exp@GLIBC_2.2.5 is picked.
Thank you very much. I'll try first with llvm 11 and if it does not
work, I'll come back with more information.
Regards,
Alexandre
Alexandre Bique via llvm-dev
So I managed to install llvm 11 on the server and got more information
thank to your help:
Projects/Podolski/CMakeFiles/Podolski.dir/__/__/AudioModulesFrame/AM_WavLoader.cpp.o:
reference to exp
Projects/Podolski/CMakeFiles/Podolski.dir/__/__/libs/tinyexpr/tinyexpr.c.o:
reference to exp
Projects/Podolski/CMakeFiles/Podolski.dir/__/__/AudioModulesFrame/AM_PitchTrack.cpp.o:
reference to exp
/usr/lib/libm.so.6: shared definition of exp@GLIBC_2.2.5
/usr/lib/libm.so.6: shared definition of exp
/usr/lib/libm.so.6: shared definition of exp@GLIBC_2.29
/usr/lib/libmvec.so.1: reference to exp@GLIBC_2.29
So the dependency is being pulled by libmvec.
Which is strange because the target is not using libmvec:
[jenkins@archear build-clang.64]$ ldd VstPlugins/libPodolski.so
linux-vdso.so.1 (0x00007ffccff47000)
libexpat.so.1 => /usr/lib/libexpat.so.1 (0x00007f2761062000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f2761040000)
libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f276103a000)
libuuid.so.1 => /usr/lib/libuuid.so.1 (0x00007f2761031000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007f2760eeb000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f2760d22000)
/usr/lib64/ld-linux-x86-64.so.2 (0x00007f2761e3e000)
[jenkins@archear build-clang.64]$
I even recompiled with -fveclib=none but it did not change anything.
And I believe that -fveclib=mvec will start working with LLVM-12.
I have no idea what pulls libmvec in.
Regards,
Alexandre Bique
Fāng-ruì Sòng via llvm-dev
Recent libm.a and libm.so are actually linker scripts
% cat /usr/lib/x86_64-linux-gnu/libm.a
/* GNU ld script
*/
OUTPUT_FORMAT(elf64-x86-64)
GROUP ( /usr/lib/x86_64-linux-gnu/libm-2.31.a
/usr/lib/x86_64-linux-gnu/libmvec.a )
% cat /usr/lib/x86_64-linux-gnu/libm.so
/* GNU ld script
*/
OUTPUT_FORMAT(elf64-x86-64)
GROUP ( /lib/x86_64-linux-gnu/libm.so.6 AS_NEEDED (
/lib/x86_64-linux-gnu/libmvec.so.1 ) )
--
宋方睿
Alexandre Bique via llvm-dev
> Recent libm.a and libm.so are actually linker scripts
>
> % cat /usr/lib/x86_64-linux-gnu/libm.a
> /* GNU ld script
> */
> OUTPUT_FORMAT(elf64-x86-64)
> GROUP ( /usr/lib/x86_64-linux-gnu/libm-2.31.a
> /usr/lib/x86_64-linux-gnu/libmvec.a )
> % cat /usr/lib/x86_64-linux-gnu/libm.so
> /* GNU ld script
> */
> OUTPUT_FORMAT(elf64-x86-64)
> GROUP ( /lib/x86_64-linux-gnu/libm.so.6 AS_NEEDED (
> /lib/x86_64-linux-gnu/libmvec.so.1 ) )
In this script the libmvec is mentioned as "AS_NEEDED", and it is not
needed in my case, yet lld pulls its dependency toward exp@GLIBC_2.29.
Is it a bug in lld?
Regards,
Alexandre
Fāng-ruì Sòng via llvm-dev
mode or (2) after --gc-sections, a symbol referenced by a live input
section is defined by the shared object.
You'll need to check whether a symbol is defined by libmvec.so.1
On Mon, Nov 9, 2020 at 9:57 AM Alexandre Bique
<bique.a...@gmail.com> wrote:
>
> On Mon, Nov 9, 2020 at 6:52 PM Fāng-ruì Sòng <mas...@google.com> wrote:
> > Recent libm.a and libm.so are actually linker scripts
> >
> > % cat /usr/lib/x86_64-linux-gnu/libm.a
> > /* GNU ld script
> > */
> > OUTPUT_FORMAT(elf64-x86-64)
> > GROUP ( /usr/lib/x86_64-linux-gnu/libm-2.31.a
> > /usr/lib/x86_64-linux-gnu/libmvec.a )
> > % cat /usr/lib/x86_64-linux-gnu/libm.so
> > /* GNU ld script
> > */
> > OUTPUT_FORMAT(elf64-x86-64)
> > GROUP ( /lib/x86_64-linux-gnu/libm.so.6 AS_NEEDED (
> > /lib/x86_64-linux-gnu/libmvec.so.1 ) )
>
> In this script the libmvec is mentioned as "AS_NEEDED", and it is not
> needed in my case, yet lld pulls its dependency toward exp@GLIBC_2.29.
>
> Is it a bug in lld?
>
> Regards,
> Alexandre
--
宋方睿
Fāng-ruì Sòng via llvm-dev
>
> No. A shared object is needed if it is linked in (1)--no-as-needed
> mode or (2) after --gc-sections, a symbol referenced by a live input
> section is defined by the shared object.
A shared object is needed if it is linked in (1) --no-as-needed mode
or (2) after --gc-sections, a symbol referenced by a live input
section is defined by the shared object as non-STB_WEAK.
Alexandre Bique via llvm-dev
I've tried various things like linking directly to libm-2.32 but it
didn't work, it seems that libm-2.32 still pulls libmvec.
I have the following linker flags:
CLANG_LDFLAGS="-fuse-ld=lld -static-libstdc++ -static-libgcc
-fvisibility=hidden -fdata-sections -ffunction-sections"
CLANG_LDFLAGS="$CLANG_LDFLAGS -Wl,--gc-sections -Wl,-O1
-Wl,--as-needed -Wl,--strip-all"
CLANG_LDFLAGS="$CLANG_LDFLAGS -Wl,-y,exp,-y,exp@GLIBC_2.29,-y,exp@GLIBC_2.2.5"
How to find what is pulling libmvec?
On the other hand, I don't understand why I have to specify that I
want exp from glibc 2.2 or 2.29, I just want the exp symbol, and if
possible the latest version available on the system.
Because I don't have a high precision requirement so they would all be
fine. Is that possible?
Regards,
Alexandre Bique
Fāng-ruì Sòng via llvm-dev
<bique.a...@gmail.com> wrote:
>
> Thank you very much for your help Fāng-ruì Sòng.
>
> I've tried various things like linking directly to libm-2.32 but it
> didn't work, it seems that libm-2.32 still pulls libmvec.
>
> I have the following linker flags:
>
> CLANG_LDFLAGS="-fuse-ld=lld -static-libstdc++ -static-libgcc
> -fvisibility=hidden -fdata-sections -ffunction-sections"
> CLANG_LDFLAGS="$CLANG_LDFLAGS -Wl,--gc-sections -Wl,-O1
> -Wl,--as-needed -Wl,--strip-all"
> CLANG_LDFLAGS="$CLANG_LDFLAGS -Wl,-y,exp,-y,exp@GLIBC_2.29,-y,exp@GLIBC_2.2.5"
>
> How to find what is pulling libmvec?
If you build a -DCMAKE_BUILD_TYPE=Debug lld, you can set a breakpoint
on lld/ELF/MarkLive.cpp:114 (ss->getFile().isNeeded) and check which
symbol makes
libmvec.so needed.
> On the other hand, I don't understand why I have to specify that I
> want exp from glibc 2.2 or 2.29, I just want the exp symbol, and if
> possible the latest version available on the system.
> Because I don't have a high precision requirement so they would all be
> fine. Is that possible?
Symbol versioning is complex ;-) While the runtime behavior is
documented by Linux Standard Base Core Specification, Generic Part,
there is no good documentation on the linker behavior.
LLD has evolved to the current state after contributors "observe GNU
ld behaviors and patch LLD when needed" :(
For an undefined symbol, there is only this form: sym@ver
For a defined symbol, both sym@ver (non-default; sometimes called
hidden) and sym@@ver (default) exist.
For a defined symbol named sym@ver, it only resolves an undefined
symbol sym@ver.
For a defined symbol named sym@@ver, it resolves both undefined
sym@ver and undefined sym.
If you have glibc 2.29, there are exp@GLIBC_2.2.5 (non-default) and
exp@@GLIBC_2.29 (default), your 'exp' undefined symbol
will resolve to exp@@GLIBC_2.29 (default) and not work with older glibc.
If you know that your translation unit does not define 'exp' (usual
case), you can write
.symver exp,exp@GLIBC_2.2.5
call exp
or
.symver exp,exp@@@GLIBC_2.2.5
call exp
Then the undefined symbol 'exp' will be renamed to 'exp@GLIBC_2.2.5'
and get bound to the old version at link time. It will thus work with
old glibc.
Alexandre Bique via llvm-dev
> > How to find what is pulling libmvec?
>
> If you build a -DCMAKE_BUILD_TYPE=Debug lld, you can set a breakpoint
> on lld/ELF/MarkLive.cpp:114 (ss->getFile().isNeeded) and check which
> symbol makes
> libmvec.so needed.
I'm afraid this is going to far for me :)
> > On the other hand, I don't understand why I have to specify that I
> > want exp from glibc 2.2 or 2.29, I just want the exp symbol, and if
> > possible the latest version available on the system.
> > Because I don't have a high precision requirement so they would all be
> > fine. Is that possible?
>
> Symbol versioning is complex ;-) While the runtime behavior is
> documented by Linux Standard Base Core Specification, Generic Part,
> there is no good documentation on the linker behavior.
> LLD has evolved to the current state after contributors "observe GNU
> ld behaviors and patch LLD when needed" :(
I think there might be some discussions needed with
glibc/coreutils/gcc/llvm because it is way too complicated.
I believe that targeting an older glibc version should not be more
complicated than
-DGLIBC_COMPAT_LEVEL=0x020205 -Wl,compat=GLIBC_2.2.5
On OSX there is a target platform command line option, and I believe
it is the right approach, yet I can't really comment on it.
> For an undefined symbol, there is only this form: sym@ver
> For a defined symbol, both sym@ver (non-default; sometimes called
> hidden) and sym@@ver (default) exist.
>
> For a defined symbol named sym@ver, it only resolves an undefined
> symbol sym@ver.
> For a defined symbol named sym@@ver, it resolves both undefined
> sym@ver and undefined sym.
>
> If you have glibc 2.29, there are exp@GLIBC_2.2.5 (non-default) and
> exp@@GLIBC_2.29 (default), your 'exp' undefined symbol
> will resolve to exp@@GLIBC_2.29 (default) and not work with older glibc.
>
> If you know that your translation unit does not define 'exp' (usual
> case), you can write
>
> .symver exp,exp@GLIBC_2.2.5
> call exp
> or
> .symver exp,exp@@@GLIBC_2.2.5
> call exp
>
> Then the undefined symbol 'exp' will be renamed to 'exp@GLIBC_2.2.5'
> and get bound to the old version at link time. It will thus work with
> old glibc.
I have one .cpp file with:
__asm__(".symver powf,powf@GLIBC_2.2.5");
__asm__(".symver expf,expf@GLIBC_2.2.5");
__asm__(".symver exp2f,exp2f@GLIBC_2.2.5");
__asm__(".symver log2f,log2f@GLIBC_2.2.5");
__asm__(".symver logf,logf@GLIBC_2.2.5");
__asm__(".symver log,log@GLIBC_2.2.5");
__asm__(".symver log2,log2@GLIBC_2.2.5");
__asm__(".symver exp,exp@GLIBC_2.2.5");
__asm__(".symver exp2,exp2@GLIBC_2.2.5");
__asm__(".symver pow,pow@GLIBC_2.2.5");
But it does not solve the issue. I've tried many variations.
Do you know if it is possible after the link, to edit the shared
library and replace the dependency on GLIBC 2.29 to GLIBC 2.2.5?
Regards,
Alexandre
Alexandre Bique via llvm-dev
Alexandre Bique via llvm-dev
llvm-objdump -sx gives:
Version References:
required from libpthread.so.0:
0x09691a75 0x00 05 GLIBC_2.2.5
0x09691972 0x00 09 GLIBC_2.3.2
0x09691973 0x00 07 GLIBC_2.3.3
0x06969192 0x00 12 GLIBC_2.12
required from libdl.so.2:
0x09691a75 0x00 21 GLIBC_2.2.5
required from libuuid.so.1:
0x09da27b0 0x00 19 UUID_1.0
required from libm.so.6:
0x09691a75 0x00 04 GLIBC_2.2.5
0x06969187 0x00 06 GLIBC_2.27
0x06969189 0x00 11 GLIBC_2.29
required from libc.so.6:
0x09691a75 0x00 03 GLIBC_2.2.5
0x0d696913 0x00 14 GLIBC_2.3
0x09691972 0x00 08 GLIBC_2.3.2
0x09691974 0x00 17 GLIBC_2.3.4
0x0d696914 0x00 16 GLIBC_2.4
0x0d696917 0x00 13 GLIBC_2.7
0x06969191 0x00 18 GLIBC_2.11
0x06969194 0x00 15 GLIBC_2.14
0x06969197 0x00 10 GLIBC_2.17
required from ld-linux-x86-64.so.2:
0x0d696913 0x00 20 GLIBC_2.3
So there are still some references to GLIBC_2.29 from libm.
It is in the VERSION_NEED or something similar I think.
Could llvm-strip figure that it can get rid of it?
Regards,
Alexandre Bique
On Wed, Nov 11, 2020 at 2:25 PM Alexandre Bique
James Y Knight via llvm-dev
Hi,
I wonder what is the right way to target an old glibc?
Fāng-ruì Sòng via llvm-dev
If you want to drop symbol versioning with llvm-objcopy:
* llvm-objcopy -R .gnu.version -R .gnu.version_r in.so out.so
* However, llvm-objcopy zeroes out the section content so at runtime glibc ld.so will error
"unsupported version 0 of Verneed record".
Thus we need to drop the DT_VERNEED tag as well.
To achieve this, we can rewrite the DT_VERNEED tag to DT_NULL (0), terminating
the dynamic table, effectively deleting all DT_VER* tags. DT_RELACOUNT is also
deleted. Fortunately its absence does not affect correctness.
This requires binary patching
...
0x000000006ffffffb (FLAGS_1) Flags: NOW
- 0x000000006ffffffe (VERNEED) 0x8ef0
- 0x000000006fffffff (VERNEEDNUM) 5
- 0x000000006ffffff0 (VERSYM) 0x89c0
- 0x000000006ffffff9 (RELACOUNT) 1536
0x0000000000000000 (NULL) 0x0
In the end, the solution is:
r2 -wqc '/x feffff6f00000000 @ section..dynamic; w0 16 @ hit0_0' a.so; llvm-objcopy -R .gnu.version -R .gnu.version_r a.so
Alexandre Bique via llvm-dev
> The correct way is to compile against a sysroot which has an old glibc (both headers and libraries!) installed in it, rather than the new glibc.
>
> Anything else you try -- such as everything being recommended in the rest of the thread -- is an unsupportable hack, and should not be done. Possibly you can get it to work, for some particular sets of glibc versions, but there can be no guarantee. (Also, this isn't really a llvm question, it's a glibc question).
Following your advice, I've tried to prepare a sysroot with various approaches:
- debootstrap ubuntu xenial and use --sysroot pointing to that
chroot; it did not work because of absolute path and symbolic link
using absolute path. And if I were to chroot in it, it would mean that
I have to replicate all the tools and their configuration (jenkins,
...) and I'd need to do more work to get a recent clang.
- use the oldest freedesktop sdk sysroot from flatpak, yet it did not
work for the same reason (absolute path and absolute symlink)
- compiling and old glibc (they don't build because -Werror catches
more problems with recent compilers than they did at the time of gcc
5)
- now I'm about to use crosstool-ng to prepare a sysroot, but I'm
afraid that I might have other surprises as it goes beyond providing a
libc.
You're correct it is now going beyond the scope of llvm-dev, so I'll
start a discussion in libc-help.
Yet I think it would be great to be able to build using a modern OS
with the latest tools and target and older OS. It is not a crazy
request isn't it?
Thank you very much,
Alexandre
Alexandre Bique via llvm-dev
> If you want to drop symbol versioning with llvm-objcopy:
>
> * llvm-objcopy -R .gnu.version -R .gnu.version_r in.so out.so
> * However, llvm-objcopy zeroes out the section content so at runtime glibc ld.so will error
> "unsupported version 0 of Verneed record".
> Thus we need to drop the DT_VERNEED tag as well.
> To achieve this, we can rewrite the DT_VERNEED tag to DT_NULL (0), terminating
> the dynamic table, effectively deleting all DT_VER* tags. DT_RELACOUNT is also
> deleted. Fortunately its absence does not affect correctness.
> This requires binary patching
>
> ...
> 0x000000006ffffffb (FLAGS_1) Flags: NOW
> - 0x000000006ffffffe (VERNEED) 0x8ef0
> - 0x000000006fffffff (VERNEEDNUM) 5
> - 0x000000006ffffff0 (VERSYM) 0x89c0
> - 0x000000006ffffff9 (RELACOUNT) 1536
> 0x0000000000000000 (NULL) 0x0
>
>
> In the end, the solution is:
>
> r2 -wqc '/x feffff6f00000000 @ section..dynamic; w0 16 @ hit0_0' a.so; llvm-objcopy -R .gnu.version -R .gnu.version_r a.so
Thank you very much for your help.
If I run:
r2 -wqc '/x feffff6f00000000 @ section..dynamic; w0 16 @ hit0_0' a.so
It results in a plugin that does not load anymore because: undefined
symbol __at_exit, version
Then if I continue with llvm-objcopy -R .gnu.version -R .gnu.version_r
a.so; then the plugins can be dlopen() but when used something goes
wrong and the plugin aborts.
I'm not certain about continuing in this direction because it is
getting very involved and if anything goes wrong difficult to support.
Thank you so much Fāng-ruì Sòng for all your help and patience.
Best regards,
Alexandre BIQUE
Alexandre Bique via llvm-dev
https://sourceware.org/pipermail/libc-help/2020-November/005555.html
Alexandre Bique