Failed to build gollvm in a docker container

[Yuan Ting]

I tried to build gollvm in a docker container (the image is based on ubuntu 20.04, and the host OS is MacOS catalina). I configured llvm by

SHELL=/bin/sh cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_ASSERTIONS=On -DLLVM_ENABLE_RTTI=On -DLLVM_USE_LINKER=gold -G Ninja ../llvm

and then build and install gollvm by 

ninja gollvm && ninja install-gollvm

after a while, I encountered

...

error: At most two relocations per offset are supported

error: At most two relocations per offset are supported

error: At most two relocations per offset are supported

[212/1279] Creating /root/llvm-project/build-debug/tools/gollvm/libgo/tmp-sigtab.go

FAILED: tools/gollvm/libgo/tmp-sigtab.go

cd /root/llvm-project/build-debug/tools/gollvm/libgo && GOARCH=amd64 GOOS=linux /bin/sh /root/llvm-project/llvm/tools/gollvm/libgo/capturescript.sh /root/llvm-project/llvm/tools/gollvm/gofrontend/libgo/mksigtab.sh /root/llvm-project/build-debug/tools/gollvm/libgo/tmp-sigtab.go

error: no SHELL setting

[214/1279] Creating /root/llvm-project/build-debug/tools/gollvm/libgo/tmp-sysinfo.go

ninja: build stopped: subcommand failed.

By add SHELL=/bin/sh, the errors above seems to be skipped but another period of time: 

...

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libgo_c_piclib.dir/__/gofrontend/libgo/go/syscall/errno.c.o: failed to match split-stack sequence at section 4 offset 0

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libgo_c_piclib.dir/__/gofrontend/libgo/go/syscall/errno.c.o: failed to match split-stack sequence at section 6 offset 0

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libgo_c_piclib.dir/__/gofrontend/libgo/go/syscall/signame.c.o: failed to match split-stack sequence at section 5 offset 0

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libbacktrace_piclib.dir/libbacktrace/backtrace.c.o: failed to match split-stack sequence at section 4 offset 0

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libbacktrace_piclib.dir/libbacktrace/backtrace.c.o: failed to match split-stack sequence at section 6 offset 0

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libbacktrace_piclib.dir/libbacktrace/dwarf.c.o: failed to match split-stack sequence at section 18 offset 0

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libbacktrace_piclib.dir/libbacktrace/dwarf.c.o: failed to match split-stack sequence at section 79 offset 0

...

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libffi_piclib.dir/libffi/src/x86/ffi64.c.o: failed to match split-stack sequence at section 22 offset 0

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libffi_piclib.dir/libffi/src/x86/ffi64.c.o: failed to match split-stack sequence at section 26 offset 0

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libffi_piclib.dir/libffi/src/x86/ffiw64.c.o: failed to match split-stack sequence at section 16 offset 0

collect2: error: ld returned 1 exit status

[901/1066] Linking C static library tools/gollvm/libgo/libgo.a

ninja: build stopped: subcommand failed.

I'm not sure the error is caused by the container environment or my configurations/other prerequisites.

[Than McIntosh]

Hello,

The ld.gold error message looks a lot like one reported in a previous bug, https://github.com/golang/go/issues/38728.

Can you please check to make sure that your client is sync'd past the CLs that fix this issue, e.g. 0edc44a02852dd0d56cac35163a45634b497d6cd.

Thanks, Than

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/23309451-cb0a-4f7b-93a1-27038187302a%40googlegroups.com.

[Yuan Ting]

Yes, I'm sure I fetched the last version of gollvm, i.e. 0edc44a02852dd0d56cac35163a45634b497d6cd

To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

[Than McIntosh]

OK.

I'm wondering if for some reason the fix I put in is not effective.

Could you please try this in your build area:

rm ./tools/gollvm/libgo/CMakeFiles/libgo_c_piclib.dir/__/gofrontend/libgo/go/syscall/errno.c.o

ninja -v libgo_all

and post the compile line for errno.c?

Thanks, Than

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1ffd3fb4-e739-4f13-9168-565e4c64bc70%40googlegroups.com.

[Yuan Ting]

Fine, that is

[1/3] /usr/bin/cc -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/gollvm/libgo -I/root/llvm-project/llvm/tools/gollvm/libgo -Iinclude -I/root/llvm-project/llvm/include -I/root/llvm-project/llvm/tools/gollvm/gofrontend/libgo/runtime -Itools/gollvm/libgo/runtime -I/root/llvm-project/llvm/tools/gollvm/libgo/libbacktrace -fPIC -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -O3    -fPIC -g -Wno-zero-length-array -fsplit-stack -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64   -UNDEBUG -MD -MT tools/gollvm/libgo/CMakeFiles/libgo_c_piclib.dir/__/gofrontend/libgo/go/syscall/errno.c.o -MF tools/gollvm/libgo/CMakeFiles/libgo_c_piclib.dir/__/gofrontend/libgo/go/syscall/errno.c.o.d -o tools/gollvm/libgo/CMakeFiles/libgo_c_piclib.dir/__/gofrontend/libgo/go/syscall/errno.c.o   -c /root/llvm-project/llvm/tools/gollvm/gofrontend/libgo/go/syscall/errno.c

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1ffd3fb4-e739-4f13-9168-565e4c64bc70%40googlegroups.com.

[Than McIntosh]

OK. Just to confirm, what do you see from 

objdump -dr ./tools/gollvm/libgo/CMakeFiles/libgo_c_piclib.dir/__/gofrontend/libgo/go/syscall/errno.c.o

for the prolog of syscall.Geterrno. Should be something like

Disassembly of section .text.syscall.GetErrno:

0000000000000000 <syscall.GetErrno>:
   0:   64 48 3b 24 25 70 00    cmp    %fs:0x70,%rsp
   7:   00 00
   9:   72 11                   jb     1c <syscall.GetErrno+0x1c>
   b:   48 83 ec 08             sub    $0x8,%rsp
   f:   e8 00 00 00 00          callq  14 <syscall.GetErrno+0x14>
                        10: R_X86_64_PLT32      __errno_location-0x4

Thanks, Than

?

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/2e9d2af9-7d78-443b-9cda-34d7b9c41623%40googlegroups.com.

[Yuan Ting]

OK, as follows

./tools/gollvm/libgo/CMakeFiles/libgo_c_piclib.dir/__/gofrontend/libgo/go/syscall/errno.c.o:     file format elf64-x86-64

Disassembly of section .text.syscall.GetErrno:

0000000000000000 <syscall.GetErrno>:

   0: f3 0f 1e fa          endbr64

   4: 64 48 3b 24 25 70 00 cmp    %fs:0x70,%rsp

   b: 00 00

   d: 72 11                jb     20 <syscall.GetErrno+0x20>

   f: 48 83 ec 08          sub    $0x8,%rsp

  13: e8 00 00 00 00        callq  18 <syscall.GetErrno+0x18>

14: R_X86_64_PLT32 __errno_location-0x4

  18: 48 63 00              movslq (%rax),%rax

  1b: 48 83 c4 08          add    $0x8,%rsp

  1f: c3                    retq

  20: 41 ba 08 00 00 00    mov    $0x8,%r10d

  26: 45 31 db              xor    %r11d,%r11d

  29: e8 00 00 00 00        callq  2e <syscall.GetErrno+0x2e>

2a: R_X86_64_PLT32 __morestack-0x4

  2e: c3                    retq

  2f: f3 0f 1e fa          endbr64

  33: eb da                jmp    f <syscall.GetErrno+0xf>

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/2e9d2af9-7d78-443b-9cda-34d7b9c41623%40googlegroups.com.

[Than McIntosh]

This looks exactly like the bug in https://github.com/golang/go/issues/38728. The prolog of the function contains

   0: f3 0f 1e fa           endbr64

which is confusing the gold linker. This instruction doesn't get added unless the host compiler has "-fcf-protection=branch" turned on.  I would suggest deleting and recreating your build area or removing your CMakeCache.txt file to insure that you have a rebuild from a clean start.

Thanks, Than

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/975b8157-5795-4c09-9aed-91278aa5f0e6%40googlegroups.com.

[eric fang]

I have been building Gollvm in a container, I can build it successfully no matter on amd64 and arm64. But the ld.gold error message really exists on both platforms.

在 2020年5月26日星期二 UTC+8上午10:14:43，Yuan Ting写道：

[Yuan Ting]

Thank you for your guidance, now I can successfully compile and link gollvm by setting CMAKE_C_FLAGS="-fcf-protection=none". But after installing gollvm, there is still a weird problem:

root@a78010e8da5d:~# go version

go: symbol lookup error: /root/llvm-project/install-release/lib64/libgo.so.11git: undefined symbol: __go_getcontext

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/975b8157-5795-4c09-9aed-91278aa5f0e6%40googlegroups.com.

[Yuan Ting]

Yes, you are right. The problems I have encountered so far are not related to the container environment.

[Than McIntosh]

>> I can successfully compile and link gollvm by setting CMAKE_C_FLAGS="-fcf-protection=none".

The manual override of  CMAKE_C_FLAGS shouldn't be needed, if you rerun cmake from a clean slate (removing the CMakeCache.txt, etc) it should detect that "-fcf-protection=branch" is hard-wired on, and will add the right flags under the hood to work around it.

>>root@a78010e8da5d:~# go version

>>go: symbol lookup error: /root/llvm-project/install-release/lib64/libgo.so.11git: undefined symbol: __go_getcontext

This symbol is defined in an assembly source file in libgo. What's happening is that by setting CMAKE_C_FLAGS="-fcf-protection=none" but not also setting CMAKE_ASM_FLAGS="-fcf-protection=none" you've introduced a discrepancy between the C and Assembly compiles in libgo. If you set CMAKE_C_FLAGS, you should also set CMAKE_ASM_FLAGS.

Thanks, Than

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/51910be4-ce4a-49a0-91f4-1e1cc63902f6%40googlegroups.com.

[Yuan Ting]

Thanks for your correction, overriding CMAKE_C_FLAGS and CMAKE_ASM_FLAGS with "-fcf-protection=none" works for me.

I also try to build gollvm from a clean slate (by recreating a build area and rerun cmake without overriding CMAKE_C_FLAGS and CMAKE_ASM_FLAGS). However, I still failed to build gollvm. I can even see the notice during the configuration:

-- trying -fcf-protection=none workaround

-- Performing Test SPLIT_STACK_WORKAROUND

-- Performing Test SPLIT_STACK_WORKAROUND - Success

-- applying -fcf-protection=none workaround

-- starting libgo configuration.

but during building, the same error happened:

[2427/2600] Linking C shared library tools/gollvm/libgo/libgo.so.11git

FAILED: tools/gollvm/libgo/libgo.so.11git

: && /usr/bin/cc -fPIC -fPIC -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -O3  -Wl,-z,nodelete -fuse-ld=gold   -Wl,-O3 -Wl,--gc-sections -shared -Wl,-soname,libgo.so.11git -o tools/gollvm/libgo/libgo.so.11git ...(omits all object files)... -Wl,-rpath,"\$ORIGIN/../lib"  -lpthread  -lm  -fsplit-stack && :

... ...

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libffi_piclib.dir/libffi/src/x86/ffi64.c.o: failed to match split-stack sequence at section 22 offset 0

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libffi_piclib.dir/libffi/src/x86/ffi64.c.o: failed to match split-stack sequence at section 26 offset 0

/usr/bin/ld.gold: error: tools/gollvm/libgo/CMakeFiles/libffi_piclib.dir/libffi/src/x86/ffiw64.c.o: failed to match split-stack sequence at section 16 offset 0

collect2: error: ld returned 1 exit status

[2429/2600] Linking C static library tools/gollvm/libgo/libgo.a

ninja: build stopped: subcommand failed.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/51910be4-ce4a-49a0-91f4-1e1cc63902f6%40googlegroups.com.

[Ivan Serdyuk]

Yuan,

do you have any progress here?

Ivan

[Yuan Ting]

Hi Ivan,

I think there is no problem to build latest gollvm in docker container now. There is no need to add "-fcf-protection" explicitly to CFLAGS/ASM_FLAGS any more according recent patches.