Slice of C types to slice of go types.

[Raul Mera]

Hey all,

I have a C function wrapped with cgo that takes a buffer allocated io, and fills it with a long array of floats (some 50K-100K numbers most of the time), so at the end I have a go slice of C.float. This so far appears to work correctly. What I actually need is a slice of float64. I guess there is no way to transform the slices without copying due to the size difference between the 2 types (It would be great if it turns out I'm wrong), but I could do a conversion on the C side and return doubles to Go. At least then the copying would be performed on the C side, which I guess would be faster (would it?). Would there be a way of transforming the slice of C doubles to a slice of float64 without copying? Also, since size is not specified in the C standard, I dont know how safe it is in practice to assume that C doubles are the same size of float64.

There is a related topic in the list, but in that case the memory was allocated by the C side, so I can hope for an easier solution in this case.

Any help will be greatly appreciated.

Cheers!

Raúl

[RoboTamer]

I think i just asked a similar question, but I also posted a awkward solution, maybe that will do for you

https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/fK8aIew83wA 

[Daniel Jo]

Well, a very unsafe way of doing this would be to use the SliceHeader type defined in the reflect package and unsafe.Pointer to manually construct a slice structure and assign the C pointer as the data. For example:

tmp := &reflect.SliceHeader{Data: dat, Len: datlen, Cap: datcap}

goslice := *((*[]float64)(unsafe.Pointer(tmp)))

Of course, this assumes that the source data is made up of C doubles and that the C array length is reported correctly. Keep in mind that the reflect package indicates that the SliceHeader type is neither safe nor portable. Further, I have no idea how nicely this plays with the garbage collector.

-Daniel

[Raul Mera]

Hey, thanks both!

I had implemented just the copying solution for the time being (github.com/rmera/gochem), which is about what RoboTamer did in his post.

I was afraid that something like Daniel's solution was the only way. I guess that what I would gain by moving the copying to the C side doesn't justify the use of reflect.SliceHeader.

Maybe a workaround would be to implement the copying concurrently, since I know the length of the array and I shouldn't even need communication except to report job completion. This way the thing would be faster than doing the copying in C, at least in some cases. 

Well, for now I can at least report, for the interested in molecular dynamics (there could be someone! :-), that Gochem is currently able to read Gromacs xtc trajectories.

Cheers!

Raúl

[John Asmuth]

http://code.google.com/p/go-wiki/wiki/cgo

[Raul Mera]

Thanks John, I actually didn't know that Wiki page.

The thing is I don't have a C array, I have a Go Slice that contains C values.

Cheers,

Raúl

[Kevin Gillette]

The thing the slice is pointing to had the same memory layout as a c of the same cap of the same size floats.

I don't see why portability is a concern, since gcc and the other modern compilers deal in ieee-754 floats, like go, so if c.sizeof(double) != 8, then iirc, the c doubles would have to have even more precision, but in any case you can panic. Otherwise the memory layout is identical, you can construct a float64 slice header with the length and cap set to `c.sizeof(thearray)/c.sizeof(double)` and the go gc will either inherit responsibility for the original alloc, or you can use finalizer's on the go side, and/or extra references to ensure the data is never gc'd, depending on what you want.

Just be sure that the original c data is not stack allocated.

[John Asmuth]

A go slice that contains C values is really easy to turn into a C array. &theSlice[0] will be the pointer to the first element in the contiguous array, and you can then treat it as a whatever* in C.

[Raul Mera]

And  all the procedure would work fine even when the memory was allocated in Go and thus it is Garbage collected?

Thanks a lot for the answers, will create  a branch to  try to implement this approach.

Cheers,

Raúl

[Kevin Gillette]

If Go allocates the memory, such as doing `data := make([]float64, 10000)`, and then passing &data[0] to a C function that takes a `double*` pointer and a length, it should work perfectly. All the above discussion, from me at least, was concerning trickiness in trying to manage data allocated by C within Go.

[Raul Mera]

The issue is that I have to allocate C doubles 'data:=make([]C.double,10000) ' and pas &data[0] to the C function. If I allocate float64, the go building  will complain (because the C function takes a *double).  Then I need to convince Go to look at the data slice as a []float64 and not as a []C.double anymore.

[Sebastien Binet]

Raul Mera <rme...@gmail.com> writes:

> The issue is that I have to allocate C doubles
> 'data:=make([]C.double,10000) ' and pas &data[0] to the C function. If I
> allocate float64, the go building will complain (because the C function
> takes a *double). Then I need to convince Go to look at the data slice as
> a []float64 and not as a []C.double anymore.

wouldn't this work ?

data := make([]float64, N)
ptr := unsafe.Pointer(&data[0])
C.frobnicate_carray((*C.double)(ptr), N)

-s

[Raul Mera]

It looks to me that I would, and it seems the obvious solution, thanks!

As soon as I get the time to implement it (it involves messing a bit with the C code as well) I'll confirm anyway.

Cheers!

Raúl

[Raul Mera]

By obvious I mean the logical one, not that it was obvious to me, of course :-)

On Tuesday, September 4, 2012 1:48:24 PM UTC+3, Sebastien Binet wrote:

[Kevin Gillette]

That's a good point -- unlike in Go, since you can always side-step type sanity in C by doing type casts, C is the place you'd want to do that. The memory layout of doubles and float64s is almost always the same, so that's the part that matters.