cgo: calling Go functions from C

[Greg Ward]

Hi all --

I'm trying to use cgo to pass a callback written in Go to a C library
using a function pointer. I was unable to get it working on my own, so
I turned to Google and found this wiki page:
http://code.google.com/p/go-wiki/wiki/cgo .

There's an example there, under "Function variables", that does not
build. Here's the example code from the wiki:

"""
package gocallback

import (
"fmt"
"unsafe"
)

/*
extern void go_callback_int(void* foo, int p1);

void CallMyFunction(void* pfoo) {
go_callback_int(pfoo, 5);
}
*/
import "C"

//export go_callback_int
func go_callback_int(pfoo unsafe.Pointer, p1 C.int) {
foo := *(*func(C.int))(pfoo)
foo(p1)
}

func MyCallback(x C.int) {
fmt.Println("callback with", x)
}

//we store it in a global variable so that the garbage collector doesn't clean up the memory for any temporary variables created.
var MyCallbackFunc = MyCallback

func Example() {
C.CallMyFunction(unsafe.Pointer(&MyCallbackFunc))
}
"""

When I try to build with "go install", here's what I get:

# cgo
/tmp/go-build474806273/cgo/_obj/example.cgo2.o: In function `CallMyFunction':
src/cgo/example.go:11: multiple definition of `CallMyFunction'
/tmp/go-build474806273/cgo/_obj/_cgo_export.o:/home/greg/src/learngo/src/cgo/example.go:11: first defined here
collect2: error: ld returned 1 exit status

Note how the silly linker is claiming that I have defined
CallMyFunction at two locations:

src/cgo/example.go:11
/home/greg/src/learngo/src/cgo/example.go:11

Yes, that is the exact same file. ;-)

Any clues?

Thanks!

Greg
--
Greg Ward http://www.gerg.ca
<gr...@gerg.ca> @gergdotca

[minux]

On Sun, Feb 10, 2013 at 7:52 AM, Greg Ward <gr...@gerg.ca> wrote:

I'm trying to use cgo to pass a callback written in Go to a C library
using a function pointer. I was unable to get it working on my own, so
I turned to Google and found this wiki page:
http://code.google.com/p/go-wiki/wiki/cgo .

this article is out-of-date, i will update it shortly. 

There's an example there, under "Function variables", that does not
build. Here's the example code from the wiki:

"""
package gocallback

import (
        "fmt"
        "unsafe"
)

/*
extern void go_callback_int(void* foo, int p1);

you don't need declare the callback, instead, you can

#include "_cgo_export.h" 

please take a look at http://tip.golang.org/cmd/cgo/ and source in misc/cgo/test.

[Maxim Khitrov]

Make CallMyFunction static. If you run 'go build -work -x example.go'
and then examine the work directory, you'll see that CallMyFunction is
defined in _cgo_export.h and also example.cgo2.c. That's the source of
the problem. Someone more familiar with cgo might be able to explain
why this happens.

- Max

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[minux]

On Sun, Feb 10, 2013 at 8:28 AM, Maxim Khitrov <m...@mxcrypt.com> wrote:

Make CallMyFunction static. If you run 'go build -work -x example.go'

then there will be more than one copy of it. best to follow the advice of docs (cited below)

and put the functions in a separate C file when using //export. 

and then examine the work directory, you'll see that CallMyFunction is
defined in _cgo_export.h and also example.cgo2.c. That's the source of
the problem. Someone more familiar with cgo might be able to explain
why this happens.

from docs of cmd/cgo: http://tip.golang.org/cmd/cgo/ (will appear in Go 1.1)

Using //export in a file places a restriction on the preamble: since it is copied into two different C output files, it must not contain any definitions, only declarations. Definitions must be placed in preambles in other files, or in C source files.

[Greg Ward]

On 10 February 2013, minux said:
> On Sun, Feb 10, 2013 at 7:52 AM, Greg Ward <gr...@gerg.ca> wrote:
>
> > I'm trying to use cgo to pass a callback written in Go to a C library
> > using a function pointer. I was unable to get it working on my own, so
> > I turned to Google and found this wiki page:
> > http://code.google.com/p/go-wiki/wiki/cgo .
> >
> this article is out-of-date, i will update it shortly.

Thanks!

> you don't need declare the callback, instead, you can
> #include "_cgo_export.h"
>
> please take a look at http://tip.golang.org/cmd/cgo/ and source in
> misc/cgo/test.

Ahh, OK, that helps. By moving one function definition to a separate
.c file, I can now build the example on the wiki page. But there's a
problem: that example, as well as the code in misc/cgo/tests, has Go
calling C code that knows it's calling back to Go.

But in the real world, C libraries just take pointers to C functions.
In my toy example, my C library is this:

// pure C; just takes a C function pointer that it calls directly
// without knowing anything about Go
int do_something(int (*callback)(int), int arg) {
printf("C: do_something(%p, %d), calling callback(%d)\n",
callback, arg, arg);
int result = callback(arg);
return result + 1;
}

An example callback written in C:

int square(int arg) {
return arg * arg;
}

and the same callback written in Go:

func square(arg C.int) C.int {
return C.int(int(arg) * int(arg))
}

I want to pass the Go implementation of square() to do_something(),
and I can't figure out how. Here's what I've got so far.

1. My Go code calls C.wrap_do_something() like this:

callback := square
callbackptr := unsafe.Pointer(&callback)
fmt.Printf("calling do_something(%v, 3)\n", callbackptr)
result := C.wrap_do_something(callbackptr, 3)
fmt.Println(result)

2. wrap_do_something() is defined in my layer of C code, on top of
do_something():

#include "_cgo_export.h"

int callgo(int arg) {
// how do I get to the "void *" that points to the Go function
// I need to call?!?
printf("C: callgo(%d): what next???\n", arg);
return 0;
}

int wrap_do_something(void *gocallback, int arg) {
// how do I get gocallback to callgo()???
return do_something(callgo, arg);
}

Note how my C function callgo() satisfies the requirements of
do_something(): it is of type "int (*callback)(int)". But since
it only takes the int that do_something() passes to it, how does
it know what to do next? Where does it get the address of the Go
function to pass back to Go code?

The obvious answer is to fix the C library so do_something() takes an
additional arbitrary void *, which it passes to its callback. But I'm
dealing with an existing, externally maintained C API (an embedded
Python interpreter). I can't change that API.

Stumped,

[Russ Cox]

A Go function pointer cannot be used directly as a C function pointer. You have to export specific Go functions, which corresponds to generating individual C wrappers, one for each function you want to call. Then you can pass the (C) function pointers for those C wrappers to other C code and it will work fine.

Russ

[Greg Ward]

Recap: I'm trying to use CGo to call a C library that then calls back
Go functions. I got past the minor problem with an out-of-date wiki
page providing code that doesn't build, and now I'm hopelessly hung up
in a thicket of function pointers. Either I am dumb or this is hard.
Too hard.

Here's the best advice I've got so far:

OK, sounds fine. But I can't get it to work.

Here's my foreign C library (not controlled by me, cannot change the
API):

// this is a C library that I cannot change; it knows nothing about
// Go, and it doesn't have a "void *userdata" hack that I can use to
// pass back-channel data in
void do_something(int (*callback)(int), int arg) {
printf("now in C do_something: callback=%p, arg=%d\n", callback, arg);
int r = callback(arg) + 5;
printf("do_something(): %d -> %d\n", arg, r);
}

Here's an example of a callback written in Go:

//export square
func square(arg C.int) C.int {
fmt.Printf("go square(%v)\n", arg)
return C.int(int(arg) * int(arg))
}

The goal: get do_something() to call square() and do something with
the result.

Attempt #1: write a wrapper, in C, for each Go callback. The C wrappers
are trivial:

int wrap_square(int arg) {
return square(arg);
}

(square() is a valid identifier because my C module has
#include "_cgo_export.h".)

Then in my Go main(), I try to pass the C wrapper to do_something():

func main() {
var cb func(C.int) C.int
var cbptr unsafe.Pointer

fmt.Println("main: calling do_something(csquare, 3)")
cb = C.wrap_square
cbptr = *(*unsafe.Pointer)(unsafe.Pointer(&cb))
C.do_something(cbptr, 3)
}

This doesn't even compile:

cb.go:30:10: must call C.wrap_square

OK, fine. You can't pass around C functions in Go. I guess that's a
limitation of CGo. Darn.

Attempt #2: write a wrapper for the unchangeable C API, pass Go
function pointers to it, and try to figure out the C function pointers
dynamically.

My Go main() now looks like this:

func main() {
var cb func(C.int) C.int
var cbptr unsafe.Pointer

fmt.Println("main: calling do_something(square, 3)")
cb = square
cbptr = *(*unsafe.Pointer)(unsafe.Pointer(&cb))
fmt.Printf(" square = %v, cb = %v, cbptr = %v\n", square, cb, cbptr)
C.wrap_do_something(cbptr, 3)
}

and the new C wrap_do_something() is:

void wrap_do_something(void *gocallback, int arg) {
int (*callback)(int);
printf("testing gocallback = %p against square = %p, triple = %p\n",
gocallback, square, triple);
if (gocallback == (void*)(square)) {
callback = wrap_square;
}
else if (gocallback == (void*)(triple)) {
callback = wrap_triple;
}
else {
fprintf(stderr,
"no C wrapper found for Go callback at %p\n", gocallback);
return;
}

do_something(callback, arg);
}

This at least compiles. But the attempt to match function pointers in
wrap_do_something() is a failure; here is the output of the resulting
program:

main: calling do_something(square, 3)
square = 0x400c00, cb = 0x400c00, cbptr = 0x400c00
testing gocallback = 0x400c00 against square = 0x4010c0, triple = 0x401110
no C wrapper found for Go callback at 0x400c00

So, back to my original question: is it actually possible to get a C
library *that has no knowledge of Go* and no "void *userdata" trick to
call Go callbacks? If, does anyone have example code that I can study?

(Someone [I believe minux] suggested misc/cgo/test/callback.go in the
Go source tree. Does not qualify: the C code there knows that it's
calling back to Go.)

Thanks --

[Maxim Khitrov]

On Wed, Feb 20, 2013 at 5:06 PM, Greg Ward <gr...@gerg.ca> wrote:
> Recap: I'm trying to use CGo to call a C library that then calls back
> Go functions. I got past the minor problem with an out-of-date wiki
> page providing code that doesn't build, and now I'm hopelessly hung up
> in a thicket of function pointers. Either I am dumb or this is hard.
> Too hard.

I'm having a hard time following your code, but perhaps the following
example will give you an idea of where to go. I recently wrote a
wrapper for SQLite, which has callback support for various events
(e.g. call a function when a transaction is committed). When
registering a callback, the API accepts a single void* that is passed
unmodified to the callback, so this allows me to keep some state. Your
example doesn't have this, so your state (the equivalent of my Conn
object) will have to be global. The rest should be pretty much the
same.

Exported functions that are called directly by SQLite:
https://code.google.com/p/go-sqlite/source/browse/go1/sqlite3/util.go#323

Macro and C functions for registering these callbacks:
https://code.google.com/p/go-sqlite/source/browse/go1/sqlite3/sqlite3.go#10

Pointers to the actual Go functions that get called from util.go (this
is the part that will need to be global in your case):
https://code.google.com/p/go-sqlite/source/browse/go1/sqlite3/sqlite3.go#130

Conn methods for registering the callbacks:
https://code.google.com/p/go-sqlite/source/browse/go1/sqlite3/sqlite3.go#411

- Max

[Russ Cox]

I think your wrap_do_something is on the right path. Instead of passing void*s, though, you can just pass an enumeration.

if(gocallback == 2)

    callback = square;

else if(gocallback == 3)

    callback = triple;

Russ