Cgo and duplicate symbols

[jim....@gmail.com]

Hi, 

I'm using two different cgo packages and ran into duplicate symbol issues.  In its simplest form, I have a source tree that looks like:

src/main/main.go

src/hello/hello.go

src/world/world.go

when I try to build main. I got duplicate symbol from linker for Print.  Any way to fixes besides renaming one of package's Print function?

Thanks,

-Jim

go build

# main

duplicate symbol _Print in:

    /var/folders/z1/5w8855qs3936qgkfkwdlztznw4qj10/T//go-link-3HJ5xB/000000.o

    /var/folders/z1/5w8855qs3936qgkfkwdlztznw4qj10/T//go-link-3HJ5xB/000001.o

ld: 1 duplicate symbol for architecture x86_64

clang: error: linker command failed with exit code 1 (use -v to see invocation)

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/6l: running clang failed: unsuccessful exit status 0x100

Here's a detail output:

main$ uname -a

Darwin jimd-macair 13.1.0 Darwin Kernel Version 13.1.0: Thu Jan 16 19:40:37 PST 2014; root:xnu-2422.90.20~2/RELEASE_X86_64 x86_64

main$ go version

go version devel +358e2b416518 Wed Mar 19 09:00:58 2014 -0700 darwin/amd64

 go build -x

WORK=/var/folders/z1/5w8855qs3936qgkfkwdlztznw4qj10/T/go-build426892071

mkdir -p $WORK/hello/_obj/

mkdir -p $WORK/

cd /Users/jim_deng/workspace/cgotest/src/hello

CGO_LDFLAGS="-g" "-O2" /Users/jim_deng/local/go/pkg/tool/darwin_amd64/cgo -objdir $WORK/hello/_obj/ -- -I $WORK/hello/_obj/ hello.go

mkdir -p $WORK/world/_obj/

cd /Users/jim_deng/workspace/cgotest/src/world

CGO_LDFLAGS="-g" "-O2" /Users/jim_deng/local/go/pkg/tool/darwin_amd64/cgo -objdir $WORK/world/_obj/ -- -I $WORK/world/_obj/ world.go

cd /Users/jim_deng/workspace/cgotest/src/hello

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/6c -F -V -w -I $WORK/hello/_obj/ -I /Users/jim_deng/local/go/pkg/darwin_amd64 -o $WORK/hello/_obj/_cgo_defun.6 -D GOOS_darwin -D GOARCH_amd64 $WORK/hello/_obj/_cgo_defun.c

clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common -print-libgcc-file-name

clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common -I $WORK/hello/_obj/ -g -O2 -o $WORK/hello/_obj/_cgo_main.o -c $WORK/hello/_obj/_cgo_main.c

cd /Users/jim_deng/workspace/cgotest/src/world

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/6c -F -V -w -I $WORK/world/_obj/ -I /Users/jim_deng/local/go/pkg/darwin_amd64 -o $WORK/world/_obj/_cgo_defun.6 -D GOOS_darwin -D GOARCH_amd64 $WORK/world/_obj/_cgo_defun.c

clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common -I $WORK/world/_obj/ -g -O2 -o $WORK/world/_obj/_cgo_main.o -c $WORK/world/_obj/_cgo_main.c

cd /Users/jim_deng/workspace/cgotest/src/hello

clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common -I $WORK/hello/_obj/ -g -O2 -o $WORK/hello/_obj/_cgo_export.o -c $WORK/hello/_obj/_cgo_export.c

cd /Users/jim_deng/workspace/cgotest/src/world

clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common -I $WORK/world/_obj/ -g -O2 -o $WORK/world/_obj/_cgo_export.o -c $WORK/world/_obj/_cgo_export.c

cd /Users/jim_deng/workspace/cgotest/src/hello

clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common -I $WORK/hello/_obj/ -g -O2 -o $WORK/hello/_obj/hello.cgo2.o -c $WORK/hello/_obj/hello.cgo2.c

cd /Users/jim_deng/workspace/cgotest/src/world

clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common -I $WORK/world/_obj/ -g -O2 -o $WORK/world/_obj/world.cgo2.o -c $WORK/world/_obj/world.cgo2.c

cd /Users/jim_deng/workspace/cgotest/src/hello

clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common -o $WORK/hello/_obj/_cgo_.o $WORK/hello/_obj/_cgo_main.o $WORK/hello/_obj/_cgo_export.o $WORK/hello/_obj/hello.cgo2.o -g -O2

cd /Users/jim_deng/workspace/cgotest/src/world

clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common -o $WORK/world/_obj/_cgo_.o $WORK/world/_obj/_cgo_main.o $WORK/world/_obj/_cgo_export.o $WORK/world/_obj/world.cgo2.o -g -O2

cd /Users/jim_deng/workspace/cgotest/src/hello

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/cgo -objdir $WORK/hello/_obj/ -dynimport $WORK/hello/_obj/_cgo_.o -dynout $WORK/hello/_obj/_cgo_import.c

cd /Users/jim_deng/workspace/cgotest/src/world

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/cgo -objdir $WORK/world/_obj/ -dynimport $WORK/world/_obj/_cgo_.o -dynout $WORK/world/_obj/_cgo_import.c

cd /Users/jim_deng/workspace/cgotest/src/hello

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/6c -F -V -w -I $WORK/hello/_obj/ -I /Users/jim_deng/local/go/pkg/darwin_amd64 -o $WORK/hello/_obj/_cgo_import.6 -D GOOS_darwin -D GOARCH_amd64 $WORK/hello/_obj/_cgo_import.c

clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common -o $WORK/hello/_obj/_all.o $WORK/hello/_obj/_cgo_export.o $WORK/hello/_obj/hello.cgo2.o -g -O2 -Wl,-r -nostdlib

cd /Users/jim_deng/workspace/cgotest/src/world

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/6c -F -V -w -I $WORK/world/_obj/ -I /Users/jim_deng/local/go/pkg/darwin_amd64 -o $WORK/world/_obj/_cgo_import.6 -D GOOS_darwin -D GOARCH_amd64 $WORK/world/_obj/_cgo_import.c

cd /Users/jim_deng/workspace/cgotest/src/hello

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/6g -o $WORK/hello.a -p hello -D _/Users/jim_deng/workspace/cgotest/src/hello -I $WORK -pack $WORK/hello/_obj/_cgo_gotypes.go $WORK/hello/_obj/hello.cgo1.go

cd /Users/jim_deng/workspace/cgotest/src/world

clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common -o $WORK/world/_obj/_all.o $WORK/world/_obj/_cgo_export.o $WORK/world/_obj/world.cgo2.o -g -O2 -Wl,-r -nostdlib

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/6g -o $WORK/world.a -p world -D _/Users/jim_deng/workspace/cgotest/src/world -I $WORK -pack $WORK/world/_obj/_cgo_gotypes.go $WORK/world/_obj/world.cgo1.go

cd /Users/jim_deng/workspace/cgotest/src/hello

pack r $WORK/hello.a $WORK/hello/_obj/_cgo_import.6 $WORK/hello/_obj/_cgo_defun.6 $WORK/hello/_obj/_all.o # internal

cd /Users/jim_deng/workspace/cgotest/src/world

pack r $WORK/world.a $WORK/world/_obj/_cgo_import.6 $WORK/world/_obj/_cgo_defun.6 $WORK/world/_obj/_all.o # internal

mkdir -p $WORK/main/_obj/

mkdir -p $WORK/main/_obj/exe/

cd /Users/jim_deng/workspace/cgotest/src/main

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/6g -o $WORK/main.a -p main -complete -D _/Users/jim_deng/workspace/cgotest/src/main -I $WORK -I /Users/jim_deng/workspace/cgotest/pkg/darwin_amd64 -pack ./main.go

cd .

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/6l -o $WORK/main/_obj/exe/a.out -L $WORK -L /Users/jim_deng/workspace/cgotest/pkg/darwin_amd64 -extld=clang $WORK/main.a

# main

duplicate symbol _Print in:

    /var/folders/z1/5w8855qs3936qgkfkwdlztznw4qj10/T//go-link-qkpMCM/000000.o

    /var/folders/z1/5w8855qs3936qgkfkwdlztznw4qj10/T//go-link-qkpMCM/000001.o

ld: 1 duplicate symbol for architecture x86_64

clang: error: linker command failed with exit code 1 (use -v to see invocation)

/Users/jim_deng/local/go/pkg/tool/darwin_amd64/6l: running clang failed: unsuccessful exit status 0x100

[Ian Lance Taylor]

On Thu, Mar 20, 2014 at 9:01 AM, <jim....@gmail.com> wrote:
>
> I'm using two different cgo packages and ran into duplicate symbol issues.
> In its simplest form, I have a source tree that looks like:
>
> src/main/main.go
> src/hello/hello.go
> src/world/world.go
>
> when I try to build main. I got duplicate symbol from linker for Print. Any
> way to fixes besides renaming one of package's Print function?

You could make the functions static.

Those are C functions, and C doesn't let you have two different global
functions with the same name.

Ian

[Jim Deng]

Thanks, Ian. I should have thought of that.  The next problem I encountered is cross compiling these two cgo packages into a go binary for android.  I'm using android's stand alone tool chain version r9d.  The error is the two cgo packages contain duplicate symbol definitions.  When I looked at the symbols, it seems to be ARM eabi functions add by toolchain gcc when generating _cgo_.o .  I am able to  reproduce it by adding a call to the eabi function in hello.go and world.go below.

src/main/main.go
src/hello/hello.go 
src/world/world.go 

Here are the command I ran:

export PATH=/Users/jim_deng/workspace/android/toolchain-r9d/bin:$PATH

CGO_CFLAGS='-I /Users/jim_deng/workspace/android/toolchain-r9d/include -march=armv7-a -mfloat-abi=softfp' \
CGO_LDFLAGS='-Wl,--fix-cortex-a8' \
CC=arm-linux-androideabi-gcc \
CXX=arm-linux-androideabi-g++ \
GOOS=linux \
GOARCH=arm \
CGO_ENABLED=1 \
go build -tags android -x

Sample errors:

/Users/jim_deng/workspace/android/toolchain-r9d/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: error: /var/folders/z1/5w8855qs3936qgkfkwdlztznw4qj10/T//go-link-8BFcZ7/000001.o: multiple definition of '__aeabi_dcmpge'

/Users/jim_deng/workspace/android/toolchain-r9d/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: /var/folders/z1/5w8855qs3936qgkfkwdlztznw4qj10/T//go-link-8BFcZ7/000000.o: previous definition here

/Users/jim_deng/workspace/android/toolchain-r9d/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: error: /var/folders/z1/5w8855qs3936qgkfkwdlztznw4qj10/T//go-link-8BFcZ7/000001.o: multiple definition of '__gtdf2'

/Users/jim_deng/workspace/android/toolchain-r9d/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: /var/folders/z1/5w8855qs3936qgkfkwdlztznw4qj10/T//go-link-8BFcZ7/000000.o: previous definition here

/Users/jim_deng/workspace/android/toolchain-r9d/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: error: /var/folders/z1/5w8855qs3936qgkfkwdlztznw4qj10/T//go-link-8BFcZ7/000001.o: multiple definition of '__gedf2

Full output:

http://hastebin.com/sodekeqibi.mel

Thanks,

-Jim

[Ian Lance Taylor]

It looks like cgo is passing libgcc.a each time it generates _all.o.
That is bringing in multiple versions of the libgcc functions, causing
this error.

Linking against libgcc was added in
http://codereview.appspot.com/5822049 for issues 1741 and 3261.
Unfortunately, the approach used there links libgcc into each package
that uses cgo. That breaks as soon as a program contains multiple
packages using cgo, and we link statically against libgcc. We need to
instead link against libgcc in the final link. I'm not sure how to
make that work with the standard library packages that use libgcc.

Please open an issue for this with your test case at
http://golang.org/issue/new . Thanks.

Ian

[Jim Deng]

Thanks for looking into this. I've opened an issue at https://code.google.com/p/go/issues/detail?id=7604&thanks=7604&ts=1395362006 .