Re: [go-nuts] Shared Library Linking with CGO

[Kyle Lemons]

This isn't really a Go question; it's a linux dynamic linking issue, and you'd have the same issue linking against that library in C.  Look up LD_LIBRARY_PATH for more help.

On Sun, Sep 16, 2012 at 8:39 PM, <rahe...@gmail.com> wrote:

Hi,

I want to use SQLite with my Golang Application on Linux and I wanted that the sqlite library be present in my own Application folder.

So firstly created a library file in a subfolder mydb :

gcc $(go env GOGCCFLAGS) -shared -o libmydb.so sqlite3.c -ldl

I am using the SQlite Binding from here :

https://github.com/mattn/go-sqlite3

I modified the CGO Line in sqlite3.go from :

#cgo pkg-config: sqlite3

To :

#cgo LDFLAGS: -L/my/src/mydb -lmydb

When I build my golang application and do a LDD :

linux-gate.so.1 =>  (0x001ad000)
libnudb.so => /usr/lib/libmydb.so (0x00786000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00d69000)
libc.so.6 => /lib/libc.so.6 (0x00b7d000)
libdl.so.2 => /lib/libdl.so.2 (0x00983000)
/lib/ld-linux.so.2 (0x0099c000)

As you can see its trying to find the Library file in :

/usr/lib/libmydb.so



Instead of looking for it in the current folder.
How can I change this so that it looks in the current folder ?

On Windows you just need to add the sqlite.dll in the same folder and it works.

Looking forward to any input.

Regards,

R

--
 
 

[Peter S]

AFAIK you cannot change it from your application.

In Linux the linker does not look at your current folder by default. You may be able to make it work by setting (or adding) "." or the actual directory path to the environment variable LD_LIBRARY_PATH.

This is equivalent to how on Linux, unlike on Windows, the shell doesn't look in your current directory for executable files by default (you have to add "." to PATH to achieve that).

Perhaps you could ship your executable with a shell script (or separate Go program) that does that.

Peter

[Ian Lance Taylor]

On Sun, Sep 16, 2012 at 8:39 PM, <rahe...@gmail.com> wrote:
>

> As you can see its trying to find the Library file in :
>
> /usr/lib/libmydb.so
>
>
> Instead of looking for it in the current folder.
>
> How can I change this so that it looks in the current folder ?

You could use
go build -ldflags="-r DIR"
where DIR is the directory you want the dynamic linker to search at runtime.

Ian

[Dave Cheney]

Loading a .so from the working directory as Windows does is a gaping
cavernous security hole. Can you please try Ian's suggestion again,
and post the output of -x if your application is still not linked
correctly.

On Mon, Sep 17, 2012 at 2:41 PM, <rahe...@gmail.com> wrote:
> Hi,
>

> I tried that and it still looks in the /usr/lib/ folder for the libmydb.so
>
> Regards,
> R

> --
>
>

[Kees Varekamp]

for sqlite you can also try static linking as described here - worked for me:

https://groups.google.com/d/msg/golang-nuts/VNP6Mwz_B6o/hhgIbxzdCQwJ

Cheers,

Kees

On Monday, September 17, 2012 6:23:44 PM UTC+12, rahe...@gmail.com wrote:

Hi,

I tried that and its still searching in the /usr/lib/ folder.

What is the -x param ?

The LD_LIBRARY_PATH solution works as intended but it will require a SHELL SCRIPT to start the application instead of it actually starting on its own.

Regards,

R

[Kees Varekamp]

no, only tried win64 actually - but that seems to work well at least

On Monday, September 17, 2012 6:57:44 PM UTC+12, rahe...@gmail.com wrote:

Hi,

I though Static linking is not at all possible at the moment.

Did you try it on Linux as well ?

Regards,

R

[Raph]

Shouldn't it be possible to use this in the #cgo part of a .go file, too?

Along the lines of:

// #cgo LDFLAGS: -r PATH
import "C"

Is this supported?

On Monday, September 17, 2012 1:36:50 PM UTC+1, rahe...@gmail.com wrote:

Hi,

I got it working now.

Thanks. The -ldflags="-r PATH" works great.

Regards,

R

On Monday, September 17, 2012 11:36:28 AM UTC+5:30, Dave Cheney wrote:

[Matt Harden]

Yes that should work and I believe it is supported. It's much better in my opinion than having to add arguments to the go build command line.

[minux]

On Sep 17, 2012 9:07 PM, "Raph" <gala...@gmx.de> wrote:
>
> Shouldn't it be possible to use this in the #cgo part of a .go file, too?
>
> Along the lines of:
>
>> // #cgo LDFLAGS: -r PATH
>> import "C"
>
> Is this supported?

no. cgo ldflags is for gcc, -ldflags is for go's linker, they are not the same thing.

[Ciprian Dorin Craciun]

On Mon, Sep 17, 2012 at 6:39 AM, <rahe...@gmail.com> wrote:
> Hi,
>
> I want to use SQLite with my Golang Application on Linux and I wanted that
> the sqlite library be present in my own Application folder.
> So firstly created a library file in a subfolder mydb :
> gcc $(go env GOGCCFLAGS) -shared -o libmydb.so sqlite3.c -ldl
>

> [...]

>
> When I build my golang application and do a LDD :
>
> linux-gate.so.1 => (0x001ad000)
> libnudb.so => /usr/lib/libmydb.so (0x00786000)
> libpthread.so.0 => /lib/libpthread.so.0 (0x00d69000)
> libc.so.6 => /lib/libc.so.6 (0x00b7d000)
> libdl.so.2 => /lib/libdl.so.2 (0x00983000)
> /lib/ld-linux.so.2 (0x0099c000)
>

> As you can see its trying to find the Library file in :
> /usr/lib/libmydb.so
>
>
> Instead of looking for it in the current folder.
>
> How can I change this so that it looks in the current folder ?

As Kyle said this is only related with Linux's `.so`-hell...

Thus what I'm saying here is not related with Go, but with Linux
in particular, and it does solve your problem quite nice...

I've seen that most of the proposed solutions relate with either:
* `LD_LIBRARY_PATH` environment variable; (which I used to apply
and gave me more problems than it solved)...
* `LDFLAGS='-r ...'` link flag, which partially solves the issue,
and brings two others:
* how do I exactly convince the build system to shovel that flag
to the linker?
* and now we're stuck with a "hard coded" path...

The best solution I could have found to the problem is the following:
* just compile your project as usual, ignoring any of the above;
* use the wonderful, but quite obscure, tool `patchelf`, to
replace the `rpath` ELF entry;
http://nixos.org/patchelf.html
* as a value to `rpath` use something like `$ORIGIN/lib`;
https://www.kernel.org/doc/man-pages/online/pages/man8/ld-linux.so.8.html
* now bundle your binary together with a folder named `lib`
containing the `.so`;

Works like a charm...
Ciprian.

[bryanturley]

Patching your binary on every build is a really bad idea. Why patch when you could just have the linker do the work in the first place.
This is like having a second linker...

I agree with Dave "Loading a .so from the working directory as Windows does is a gaping cavernous security hole."

Normally if you are giving your app to some consumer you would have it packaged in a .rpm/.deb/other with proper dependency checking.
In that case you could distribute your custom built .so files in another .rpm/.deb file and have it installed in the proper place and tracked with the distributions normal utilites.
Best not to replace the distributions libs if it already has them as well, rename yours if it is a popular library or just use the distros. 
Makes a sysadmin's life a lot easier when you play by the rules, and it really isn't that hard to follow the standard.

[minux]

I advocate static linking.

If you bundle with your application a specific version of libsqlite3.so, why not just

statically link it into your application? (just put the amalgamation .c and .h in sqlite3

driver directory and rebuild the driver)

Then you can avoid dll/so-hell, avoid security holes, and also won't mess with

system libraries in /lib or /usr/lib.

I know this is against the "modern" practice of dynamic linking almost everything,

but Go is static linking in the first place.

[bryanturley]

On Tuesday, October 9, 2012 2:30:00 PM UTC-5, minux wrote:

I advocate static linking.

I static link the little bit of cgo code I use as well, it is more inline with KISS.
The only problem would be static linking gpl (or similar code) into your non-gpl program, but that is off topic and I hesitate to send a message with this line in it.....

[Ciprian Dorin Craciun]

On Tue, Oct 9, 2012 at 10:16 PM, bryanturley <bryan...@gmail.com> wrote:
> Patching your binary on every build is a really bad idea. Why patch when you
> could just have the linker do the work in the first place.
> This is like having a second linker...

Don't get me wrong: I'm not suggesting patching every build...

I was suggesting only this: if you find that you need (for some
reason) to ship your Go code somewhere where you don't have your
needed libraries (for some reason) then `patchelf` is the least
troublesome solution: just tar, untar, execute.

> Normally if you are giving your app to some consumer you would have it
> packaged in a .rpm/.deb/other with proper dependency checking.

For a product yes, you do the packaging dance. But for a small
toy, a prototype? It would take me more time to learn how to package
than how to write it...

> Makes a sysadmin's life a lot easier when you play by the rules, and it
> really isn't that hard to follow the standard.

:) Which standard? I mean which one of the few dozen? (Only to
count the linuces.)

Ciprian.

[Ciprian Dorin Craciun]

On Tue, Oct 9, 2012 at 10:29 PM, minux <minu...@gmail.com> wrote:
> I advocate static linking.
>

> I know this is against the "modern" practice of dynamic linking almost
> everything,
> but Go is static linking in the first place.

I am totally for that. Actually one of the Go's selling point for
me was this.

> If you bundle with your application a specific version of libsqlite3.so, why
> not just
> statically link it into your application? (just put the amalgamation .c and
> .h in sqlite3
> driver directory and rebuild the driver)

You can't do that for every library? (Because it is still compiled
with something other than `gcc`, right?)

[Ciprian Dorin Craciun]

On Tue, Oct 9, 2012 at 10:47 PM, bryanturley <bryan...@gmail.com> wrote:
> On Tuesday, October 9, 2012 2:30:00 PM UTC-5, minux wrote:
>>
>> I advocate static linking.
>
>
> I static link the little bit of cgo code I use as well, it is more inline
> with KISS.

Is this possible yet? (I mean to statically link a Go application
that uses CGO.) I haven't seen it in the changelog / release notes...

> The only problem would be static linking gpl (or similar code) into your
> non-gpl program, but that is off topic and I hesitate to send a message with
> this line in it.....

(By my knowledge even if you dynamically link against GPL you must
apply GPL to your code...)

[Tim Shannon]

For me at least, this would be the ideal solution, but the problem is some of the libraries I need are C++ with C headers, and not C, so they can't be compiled by CGO.

 

On Tuesday, October 9, 2012 2:30:00 PM UTC-5, minux wrote:

[minux]

On Thu, Oct 11, 2012 at 12:04 AM, Tim Shannon <shannon...@gmail.com> wrote:

For me at least, this would be the ideal solution, but the problem is some of the libraries I need are C++ with C headers, and not C, so they can't be compiled by CGO.

You will be delighted when issue 4069 is fixed. It will solve at least the problem of

linking with C++ code statically.