SWIG/CGO and return-by-reference strings

[mark mellar]

Hi folks!

So, I'm trying to integrate my lovely Go code with an evil 3rd party C api. I've actually gotten pretty far using just SWIG to generate wrappers but the whole C-string thing is starting to melt my brain. Could someone dumb this down to the level of an eight year old for me?

So, here's what I'm trying to do...

First the prototype of the C function I'm trying to call.

a_ReturnCode a_c_function(const char* inputString1, const char*

 inputString2, char** outputString1, char** outputString2,

 char** outputString3, bool inputBool1 = false, const char* inputString3 = NULL);

I'm using SWIG to generate a wrapper for my Go code to call, which looks like this...

inStr1 := "foo"+string('\x00')

inStr2 := "bar"+string('\x00')
inStr3 := "baz"+string('\x00')

var outStr1, outStr2, outStr3 string

ret := A_c_function(inStr1, inStr2, &outStr1, &outStr1, &outStr1, false, inStr3)

log.Infof("Got return code '%d', message '%s', uuid '%s', name '%s'", ret, outStr1, outStr2, outStr3)

Now, of course this doesn't work as my output strings don't have any memory allocated to them. I can think of a couple of possible solutions, none of which I'm particularly enthusiastic about. Using CGO to call malloc, then converting to a go string later, or using make to create an array of C.char and passing in a pointer to the first element. The fact that the C function requires char** rather than just char* also has me a bit stumped, but that's a question for another time and place...

My question to you guys is what do you think the best way to do this is, and why?

Cheers!

Mark.

[Tamás Gulácsi]

See http://blog.golang.org/c-go-cgo

inStr1 := C.CString("foo")

defer C.free(unsafe.Pointer(inStr1))

inStr2 := C.CString("bar")

defer C.free(unsafe.Pointer(inStr2))
inStr3 := C.CString("baz")

defer C.free(unsafe.Pointer(inStr3))

out1 := make([]byte, 1024)

out2 := make([]byte, 1024)

out3 := make([]byte, 1024)

ret := A_c_function(inStr1, inStr2, &out1[0], &out2[1], &out3[0], false, inStr3)

outStr1 := C.GoString(out1)

outStr2 := C.GoString(out2)

outStr3 := C.GoString(out3)

[mark mellar]

Thanks!

Turns out my swig-generated wrapper is asking for *string rather than *byte/*C.char though, so this doesn't quite work. I've read through he cgo blog but haven't found anything I can twist into a solution to my problem (likely says more about me than the blog post!). I'll continue hacking around til I find something that works...

M 

[Ian Lance Taylor]

On Thu, Feb 11, 2016 at 11:28 AM, mark mellar <radiat...@gmail.com> wrote:
>
> Turns out my swig-generated wrapper is asking for *string rather than
> *byte/*C.char though, so this doesn't quite work. I've read through he cgo
> blog but haven't found anything I can twist into a solution to my problem
> (likely says more about me than the blog post!). I'll continue hacking
> around til I find something that works...

You can change what the SWIG wrapper expects by using typemaps. See
the SWIG docs and the example at
https://github.com/swig/swig/blob/master/Examples/test-suite/go_inout.i
.

Ian

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[mark mellar]

Thanks for your help folks,

In case anyone is wondering how I solved this, here's an example (feel free to comment/critisise if you think I could have done better!)

First, a simple C API to recreate my requirement...

//test.c

#include <stdlib.h>

#include <string.h>

#include <stdio.h>

int get_string(char** return_string){

char* myBuff = (char*) malloc(sizeof(char) * 15);

strcpy(myBuff, "Hello World");

    *return_string = myBuff;

    return 42;

}

//test.h

extern int get_string(char** return_string);

The API takes a pointer to a string (char**) from the caller and points it towards a string it puts in heap memory.

Now, by default SWIG will map a C char** to a Go *string. This doesn't work, as when Go tries to dereference *string it finds a C char*, which it isn't expecting (I think).

//main.go
package main

import "fmt"

import "stringtest"

func main() {

strPtr := new(string)

retInt := stringtest.Get_string(&strPtr)

fmt.Printf("'%d', '%s'\n", retInt, *strPtr)

}

Results in...

'42', ''

To get around this I used Go's C package, a couple of helper functions and a SWIG typemap

%module stringtest

%include<typemaps.i>

%{

#include "test.h"

%}

%go_import("C")

%insert(go_header)

%{

type StringReturner **C.char

func NewStringReturner() StringReturner {

ptr := new(C.char)

return &ptr

}

func Return(this StringReturner) string {

str := C.GoString(*this)
        C.free(*this)
        C.free(this)

return str

}

%}

%typemap(gotype) char** "StringReturner"

int get_string(char **return_string);

So, I've defined a new type 'StringReturner'  which is essentially a **C.char (essentially identical to C's char**(?)) with functions to construct the type, and to resolve it back to a go string (cleaning up allocated memory as it goes). The %typemap line tells SWIG to map C's char** to my new type.

I can then call into the API like this...

package main

import "C"

import "fmt"

import "stringtest"

func main() {

strPtr := stringtest.NewStringReturner()

retInt := stringtest.Get_string(strPtr)

fmt.Printf("'%d', '%s'\n", retInt, stringtest.Return(strPtr))

}

Which results in...

'42', 'Hello World'

I know there are probably better ways of driving swig to achieve this without adding/managing the extra types, but this is the best I've got working so far and I don't have any more time to spend learning the intricacies of SWIG's type mapping system so this'll have to do for now.

Hope this helps others with similar issues to get something working!

Thanks again!

Mark.

[mark mellar]

Of course, the cgo updates in 1.6 break this

panic: runtime error: cgo argument has Go pointer to Go pointer

Looks like we'll have to hold off updating until I figure out a better solution :(

M

[Tamás Gulácsi]

Instead of
ptr := new(C.char)

Use
var arr [1]C.char
ptr := &arr[0]

Or
ptr := (*C.char)(unsafe.Pointer(C.malloc(C.sizeof_void)))