How to export C variables from Go shared libraries?

315 views
Skip to first unread message

Hiroaki Nakamura

unread,
Jul 16, 2015, 9:47:37 AM7/16/15
to golang-nuts
Hi gophers,

Can I export C variables from Go shared libraries?
I would like to write an Apache or nginx modules in Go.

I would like to export module variables like

If it is not possible to export C variables, is there a workaround for
writing Apache or nginx modules in Go?

Thanks in advance!

--
Hioraki Nakamura )hnak...@gmail.com)

Ian Lance Taylor

unread,
Jul 16, 2015, 5:27:02 PM7/16/15
to Hiroaki Nakamura, golang-nuts
On Thu, Jul 16, 2015 at 6:46 AM, Hiroaki Nakamura <hnak...@gmail.com> wrote:
>
> Can I export C variables from Go shared libraries?

You can only use the //export comment with functions.

However, you can put variables in the C header. For example, you can
write

// int i;
import "C"


func init() {
C.i = 1
}

Then uses of the shared library can refer to i.

At least with the current implementation: there is a catch: the init
function will only be reliably run after some function in the file is
called, because the initialization runs in a separate thread. So
although your C code can refer to the variable, it may not be
initialized unless you can ensure that some Go function is called.

Ian

Hank Donnay

unread,
Jul 16, 2015, 5:34:51 PM7/16/15
to Ian Lance Taylor, Hiroaki Nakamura, golang-nuts
Would something like

    //export libInit
    func() libInit() {}

do the job, or might the compiler do deep magjicks and not call init() on an empty function?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
-hank

Ian Lance Taylor

unread,
Jul 16, 2015, 7:07:40 PM7/16/15
to Hank Donnay, Hiroaki Nakamura, golang-nuts
On Thu, Jul 16, 2015 at 2:34 PM, Hank Donnay <hdo...@gmail.com> wrote:
>
> Would something like
>
> //export libInit
> func() libInit() {}
>
> do the job, or might the compiler do deep magjicks and not call init() on an
> empty function?

I'm not sure I understand the suggestion. Is there a system for which
a function named libInit is called automatically when a shared library
is loaded? That is not the case on GNU/Linux.

Ian

Hank Donnay

unread,
Jul 16, 2015, 9:21:03 PM7/16/15
to Ian Lance Taylor, Hiroaki Nakamura, golang-nuts
No, just suggesting a "manual" init function for a library that doesn't do anything but trigger's the go package's init() to run.
--
-hank

Ian Lance Taylor

unread,
Jul 16, 2015, 11:09:04 PM7/16/15
to Hank Donnay, Hiroaki Nakamura, golang-nuts
On Thu, Jul 16, 2015 at 6:20 PM, Hank Donnay <hdo...@gmail.com> wrote:
> No, just suggesting a "manual" init function for a library that doesn't do
> anything but trigger's the go package's init() to run.

Sure, that will work.

Hiroaki Nakamura

unread,
Jul 17, 2015, 11:09:36 AM7/17/15
to Ian Lance Taylor, Hank Donnay, golang-nuts
Thanks for your advices.

I made a github project and did an experiment on Linux and OSX.

I tested two versions:

* Explicitly calling libInit():
* On OSX, I got a link error:
  ld: 1 duplicate symbol for architecture x86_64

I have two questions:

* How can I a fix a segmentation fault while calling dlclose()?
* Is it possible to running init() without calling libInit()?

Thanks!

--
Hioraki Nakamura )hnak...@gmail.com)

Fredrik Ehnbom

unread,
Jul 17, 2015, 11:46:32 AM7/17/15
to golan...@googlegroups.com, hdo...@gmail.com, ia...@golang.org
>  init() without calling libInit()?

init will run without calling libInit; try for example adding usleep(1000*1000) before printing the variable the first time. Calling the dummy libInit function (or any other go function) just guarantees that all init functions have finished.

Hiroaki Nakamura

unread,
Jul 17, 2015, 12:49:03 PM7/17/15
to Fredrik Ehnbom, golang-nuts
Hi Fredrik,
Thanks for your comment.

I tried an example, dlopen, usleep(1000*1000), then referencing i:
https://github.com/hnakamur/export_c_variable_from_go_dll_experiment/tree/1951dfb411dcff8986687ca98f801505397fc6cf

The result summary is:

* dlopen example on Linux
  * i is set to 1 after dlopen then sleep.
  * dlclose now runs successfully without segmentation fault.
* compile time link example on Linux
  * i remains 0
* on osx
   * link error: ld: 1 duplicate symbol for architecture x86_64

I have another question:
How can I know the go initialization thread finishes from C?

I welcome further comments.
Thanks!



--
Hioraki Nakamura )hnak...@gmail.com)

Ian Lance Taylor

unread,
Jul 17, 2015, 6:49:48 PM7/17/15
to Hiroaki Nakamura, Fredrik Ehnbom, golang-nuts
On Fri, Jul 17, 2015 at 9:47 AM, Hiroaki Nakamura <hnak...@gmail.com> wrote:
>
> * dlclose now runs successfully without segmentation fault.

I would not count on dlclose working.

> How can I know the go initialization thread finishes from C?

Currently you can't. Calling a Go function waits for the
initialization thread to complete, but there is no way to wait without
calling a Go function. Please open an issue for this. Thanks.

Ian

Hiroaki Nakamura

unread,
Jul 17, 2015, 7:56:26 PM7/17/15
to Ian Lance Taylor, Fredrik Ehnbom, golang-nuts
2015-07-18 7:49 GMT+09:00 Ian Lance Taylor <ia...@golang.org>:
On Fri, Jul 17, 2015 at 9:47 AM, Hiroaki Nakamura <hnak...@gmail.com> wrote:
>
>   * dlclose now runs successfully without segmentation fault.

I would not count on dlclose working.

Could you tell me why, please?
I thought dlclose is working because "after dlclose" is printed
 

> How can I know the go initialization thread finishes from C?

Currently you can't.  Calling a Go function waits for the
initialization thread to complete, but there is no way to wait without
calling a Go function.  Please open an issue for this.  Thanks.

Alright. I opened an issue at https://github.com/golang/go/issues/11768


Now what to do about these two problems?


* compile time link example on Linux
  * i remains 0
* on osx
   * link error: ld: 1 duplicate symbol for architecture x86_64

Thanks!

--
Hioraki Nakamura )hnak...@gmail.com)

Ian Lance Taylor

unread,
Jul 17, 2015, 10:40:34 PM7/17/15
to Hiroaki Nakamura, Fredrik Ehnbom, golang-nuts
On Fri, Jul 17, 2015 at 4:55 PM, Hiroaki Nakamura <hnak...@gmail.com> wrote:
> 2015-07-18 7:49 GMT+09:00 Ian Lance Taylor <ia...@golang.org>:
>>
>> On Fri, Jul 17, 2015 at 9:47 AM, Hiroaki Nakamura <hnak...@gmail.com>
>> wrote:
>> >
>> > * dlclose now runs successfully without segmentation fault.
>>
>> I would not count on dlclose working.
>
> Could you tell me why, please?
> I thought dlclose is working because "after dlclose" is printed
> with code at
> https://github.com/hnakamur/export_c_variable_from_go_dll_experiment/blob/1951dfb411dcff8986687ca98f801505397fc6cf/runtime_load.c#L18

The Go runtime starts threads on which it runs goroutines. If any of
those threads are running when you call dlclose, your program is
likely to crash. Currently there is no way to cleanly shut down the
Go runtime.


> Now what to do about these two problems?
>
> * compile time link example on Linux
> * i remains 0

This is the problem we have been talking about, isn't it?


> * on osx
> * link error: ld: 1 duplicate symbol for architecture x86_64

I don't know. Is there an issue for this one?

Ian

Hiroaki Nakamura

unread,
Jul 18, 2015, 12:10:33 AM7/18/15
to Ian Lance Taylor, Fredrik Ehnbom, golang-nuts
2015-07-18 11:40 GMT+09:00 Ian Lance Taylor <ia...@golang.org>:
On Fri, Jul 17, 2015 at 4:55 PM, Hiroaki Nakamura <hnak...@gmail.com> wrote:
> 2015-07-18 7:49 GMT+09:00 Ian Lance Taylor <ia...@golang.org>:
>>
>> On Fri, Jul 17, 2015 at 9:47 AM, Hiroaki Nakamura <hnak...@gmail.com>
>> wrote:
>> >
>> >   * dlclose now runs successfully without segmentation fault.
>>
>> I would not count on dlclose working.
>
> Could you tell me why, please?
> I thought dlclose is working because "after dlclose" is printed
> with code at
> https://github.com/hnakamur/export_c_variable_from_go_dll_experiment/blob/1951dfb411dcff8986687ca98f801505397fc6cf/runtime_load.c#L18

The Go runtime starts threads on which it runs goroutines.  If any of
those threads are running when you call dlclose, your program is
likely to crash.  Currently there is no way to cleanly shut down the
Go runtime.

I see. Thanks for your expression.
 
> Now what to do about these two problems?
>
> * compile time link example on Linux
>   * i remains 0

This is the problem we have been talking about, isn't it?

No, it is a different problem.

```
#include <stdio.h>
#include <dlfcn.h>
#include "libgodll.h"

int main(int argc, char **argv) {
printf("compiletime_load started\n");
usleep(1000 * 1000);
printf("i=%d\n", i);
return 0;
}
```

The code above prints `i=0` instead of `i=1`.
> * on osx
>    * link error: ld: 1 duplicate symbol for architecture x86_64

I don't know.  Is there an issue for this one?

I think there is none.
I searched issues for "is:issue is:open duplicate symbol"
https://github.com/golang/go/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+duplicate+symbol

--
Hioraki Nakamura )hnak...@gmail.com)

Ian Lance Taylor

unread,
Jul 18, 2015, 1:37:49 AM7/18/15
to Hiroaki Nakamura, Fredrik Ehnbom, golang-nuts
On Fri, Jul 17, 2015 at 9:09 PM, Hiroaki Nakamura <hnak...@gmail.com> wrote:
>
> No, it is a different problem.
>
> https://github.com/hnakamur/export_c_variable_from_go_dll_experiment/blob/master/compiletime_load.c
>
> ```
> #include <stdio.h>
> #include <dlfcn.h>
> #include "libgodll.h"
>
> int main(int argc, char **argv) {
> printf("compiletime_load started\n");
> usleep(1000 * 1000);
> printf("i=%d\n", i);
> return 0;
> }
> ```
>
> The code above prints `i=0` instead of `i=1`.

Yes. You need to call a Go function in order for initialization
functions to run. Calling sleep is not reliable.


>> > * on osx
>> > * link error: ld: 1 duplicate symbol for architecture x86_64
>>
>> I don't know. Is there an issue for this one?
>
>
> I think there is none.
> I searched issues for "is:issue is:open duplicate symbol"
> https://github.com/golang/go/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+duplicate+symbol

I don't think any of those issues are for the same thing.

It may be that GCC and clang have different defaults for whether "int
i;" is a common symbol. It may be necessary to use something like

#cgo CFLAGS "-fcommon"

assuming that works with clang.

Ian

Hiroaki Nakamura

unread,
Jul 18, 2015, 2:26:32 AM7/18/15
to Ian Lance Taylor, Fredrik Ehnbom, golang-nuts
2015-07-18 14:37 GMT+09:00 Ian Lance Taylor <ia...@golang.org>:
On Fri, Jul 17, 2015 at 9:09 PM, Hiroaki Nakamura <hnak...@gmail.com> wrote:
>
> No, it is a different problem.
>
> https://github.com/hnakamur/export_c_variable_from_go_dll_experiment/blob/master/compiletime_load.c
>
> ```
> #include <stdio.h>
> #include <dlfcn.h>
> #include "libgodll.h"
>
> int main(int argc, char **argv) {
> printf("compiletime_load started\n");
> usleep(1000 * 1000);
> printf("i=%d\n", i);
> return 0;
> }
> ```
>
> The code above prints `i=0` instead of `i=1`.

Yes.  You need to call a Go function in order for initialization
functions to run.  Calling sleep is not reliable.

I did another experiment with calling the Go function libInit() explicitly.
However i still remains even after calling libInit() followed by usleep().


#include <stdio.h>
#include <dlfcn.h>
#include "libgodll.h"

int main(int argc, char **argv) {
printf("compiletime_load started\n");
printf("#1 i=%d\n", i);
libInit();
printf("#2 i=%d\n", i);
usleep(1000 * 1000);
printf("#3 i=%d\n", i);
return 0;
}
```
>> > * on osx
>> >    * link error: ld: 1 duplicate symbol for architecture x86_64

It may be that GCC and clang have different defaults for whether "int
i;" is a common symbol.  It may be necessary to use something like

#cgo CFLAGS "-fcommon"

assuming that works with clang.

That did the trick! Thanks for your advice.

I added the line below:
//#cgo CFLAGS: "-fcommon"
https://github.com/hnakamur/export_c_variable_from_go_dll_experiment/commit/c5092cea2dc7b4ae5009392ec8b1aa7529bdf166#diff-1

--
Hioraki Nakamura )hnak...@gmail.com)

Ian Lance Taylor

unread,
Jul 18, 2015, 1:55:48 PM7/18/15
to Hiroaki Nakamura, Fredrik Ehnbom, golang-nuts
Ah, yes, you're right. Sorry.

It's because Go shared libraries are created using -Bsymbolic. That
causes the i in the Go library to be different from the i in the main
executable. This won't matter if you use dlopen and dlsym to access
the variable. But it will matter if you use a simple "extern int i;"
and link against the shared library directly.

Unfortunately, at present, I don't think there is any reliable way for
a Go shared library to export a variable to C code.

Ian

Hiroaki Nakamura

unread,
Jul 18, 2015, 6:48:25 PM7/18/15
to Ian Lance Taylor, Fredrik Ehnbom, golang-nuts

OK, now I am clear about the current situation.
Thank you very much for your assistance!

Best regards, 
Hioraki Nakamura
Reply all
Reply to author
Forward
0 new messages