Creating C++ bindings to go package structs

255 views
Skip to first unread message

Justin Israel

unread,
May 6, 2016, 1:26:39 AM5/6/16
to golang-nuts
Hi,

When I had first started approaching the task of making C++ bindings around a go library, I hit some walls and asked about it on StackOverflow. I got no replies, and ended up getting a working solution. However, I wanted to check this approach with the community to see if I could have done this a better way.

package gofileseq has 2 structs and a few functions that I wanted to make available as a C++ library. Since the cgo rules disallowed C++ storing pointers for undetermined periods of time, and also passing memory containing pointers at all, I had to figure out how to make objects that contain state accessible to C++. 

What I ended up doing was to create a private package level map, where I associate instances of the objects with a unique uint64 id, and then give that to C++ as an opaque handle: 

I also use a reference counting approach, so that C++ can copy and destruct and ultimately trigger the removal of these objects, by id:

Then all of the wrapper methods in C++ simply delegate to Go exported functions, using the id:

This all works. However, there is a mutex guarded map now, around all object access. It uses a write lock for object create/delete, and a read lock for method access. 

Some random performance tests show, when creating 100k FileSequence instances and calling 4 different getters, I get:

Python (original library):  8.0s
Go (port from python):    1.9s
C++ bindings over Go:    3.2s

So all that being said, is there a way I could have approached this binding without a global map? It seems really straightforward to export plain data to C/C++ but when it comes to exposing functionality for objects that have state and methods, I couldn't come up with anything better.

Much appreciated for any insight! It was a really interesting learning experience so far.

justin



Ian Lance Taylor

unread,
May 6, 2016, 11:58:08 AM5/6/16
to Justin Israel, golang-nuts
I didn't look at your code, but using a map on the Go side to generate
a handle is a typical approach to this kind of issue.

Another approach, that avoids the global lock on the map, is to
allocate your memory using C.malloc. Then you get a C pointer that
you can safely return to C, probably as a uintptr. Since the C++ code
is managing the lifespan of your objects, you aren't getting any real
benefit from Go garbage collection, so allocating in C memory makes
sense. Whether this would work for your use case I do not know.

Ian

Justin Israel

unread,
May 6, 2016, 5:05:37 PM5/6/16
to Ian Lance Taylor, golang-nuts
This is the first approach I wanted to go with, but what does that mean for exposing a structure that has internal state and holds a reference to another structure, which also has state? Does it mean I would need to break apart the internals into data that can be stored in malloc'd memory in C? Or is there actually a way to allocate the Go struct into C memory as-is? It would be ideal if C++ could maintain the lifetime of the Go struct, in-tact. 
I had seen another StackOverflow question where someone suggested they did have to pass around a handful of pointers that comprised the Go object.
 

Ian

Ian Lance Taylor

unread,
May 6, 2016, 5:19:50 PM5/6/16
to Justin Israel, golang-nuts
I don't think I quite understand what you are asking.

To use the C memory allocator you basically have to allocate
everything just as you would in C.
p := (*GoType)unsafe.Pointer(C.malloc(unsafe.Sizeof(GoType{})))

Ian

Justin Israel

unread,
May 6, 2016, 5:29:24 PM5/6/16
to Ian Lance Taylor, golang-nuts
Actually, you understood it perfectly :-)
This looks like what I was after. So this means I could have my FileSequence and FrameSet instances unmanaged by Go, and just pass the unsafe.Pointer between C and cast back and forth. If this works as I hope, then I much prefer it to a global map!

I will try it out. Thanks so much!
 

Ian

Justin Israel

unread,
May 7, 2016, 1:39:12 AM5/7/16
to Ian Lance Taylor, golang-nuts
I started looking at this approach and realized it is a bit more invasive to the way the Go library works, because as far as I understand it I would have to break apart the structs and control the allocation of the internal data to be malloc'd instead of through Go.
type InclusiveRange struct {
    start int
    end   int
    step  int

    cachedEnd   int
    isEndCached bool

    cachedLen   int
    isLenCached bool
}
//...
type InclusiveRanges struct {
    blocks []*InclusiveRange
}
//...
type FrameSet struct {
    frange   string
    rangePtr *ranges.InclusiveRanges
}
//... 
type FileSequence struct {
    basename string
    dir      string
    ext      string
    padChar  string
    zfill    int
    frameSet *FrameSet
}

So I would start by malloc'ing a FileSequence and then filling out the fields. But because creating a NewFileSequence() also builds the *FrameSet field through a NewFrameSet(), I would need to break apart its constructor function so that I can malloc the FrameSet instead and copy over its values. But the FrameSet uses a slice of InclusiveRange objects, which also get allocated through Go. 

Ian, is this basically how you would see this having to work, if using the malloc approach? I would need to make sure I malloc all the way through the structure, to be able to avoid GC and let C++ manage it. I've started by refactoring my NewFileSequence() so that I have a package private function that can fill an existing empty *FileSequence. I would use this to pass a C malloc'd struct and have the fields filled.
 
 

Ian
Reply all
Reply to author
Forward
0 new messages