cgo iterating through C memory block

[Nick Sarten]

Hi there,

I'm trying to make an interface between OpenCL and Go at the moment,
but i keep running into roadblocks caused by incompatibilities when
converting between Go and C types, especially when it comes to arrays.
Consider the following block of code i have written:

func (c *Context) Devices() ([]Device, os.Error) {
var num_devices C.size_t
var err C.cl_int = C.clGetContextInfo(c.c_context,
C.CL_CONTEXT_DEVICES, C.size_t(0), nil, &num_devices)
if err != C.CL_SUCCESS {
return nil, os.NewError("An error has occurred while trying to find
the number of drivers attached to a context: " +
strconv.Itoa(int(err)))
}
device_ids := C.malloc(C.size_t(unsafe.Sizeof(new(C.cl_device_id)) *
int(num_devices)))
defer C.free(device_ids)
err = C.clGetContextInfo(c.c_context, C.CL_CONTEXT_DEVICES,
num_devices, device_ids, &num_devices)
if err != C.CL_SUCCESS {
return nil, os.NewError("An error has occurred while trying to
retrieve a list of drivers attached to a context: " +
strconv.Itoa(int(err)))
}
num_devices_int := int(num_devices)
device_ids_go := (*[]C.cl_device_id)(device_ids)

//create a device object for each device and set it's id
devices := make([]Device, num_devices_int)
for i := 0; i < num_devices_int; i++ {
devices[i].id = unsafe.Pointer(&(*device_ids_go)[i])
}
return devices, nil
}

I have a few questions regarding how to iterate through a memory block
such as the one i malloc'd in the code above. As you can see I have
attempted to cast the pointer to the start of the memory block to a
slice of some sort. This allows me to access the first item in the
array, but subsequent items are nil.

device_ids_go := (*[]C.cl_device_id)(device_ids)

This was originally suggested to me in the go-nuts irc channel, but it
seems that it simply converts the pointer to a length 1 slice. Is
there any way to make this work, or even a different way to cycle
through the memory block? When i asked this on the irc channel most
recently I got some answers about casting to a huge fixed length array
and slicing (which seemed hacky), or trying to create a slice header
manually. This seems like a fairly common case of C usage, so i'l
hoping there's an easier way to handle it that i've just overlooked or
not been able to find.

Thanks in advance for any assistance.

[Evan Shaw]

The suggestions about manually creating a slice header or creating a
large fixed-size array are the standard ways to do it.

The fixed-size array approach is slightly easier and requires less
knowledge of how things work internally, but it also doesn't let you
set the slice's capacity correctly, which could result in some pretty
big problems depending on how the slice is used. It'd look something
like this:

device_ids_go = (*[1<<20]C.cl_device_id)(device_ids)[:num_devices]

The slice header approach is safer, but a little more involved:

var header reflect.SliceHeader
header.data = uintptr(device_ids)
header.len = num_devices
header.cap = num_devices
device_ids_go = *(*[]C.cl_device_id)(unsafe.Pointer(&header))

I might have gotten the conversions wrong, but the real versions
should be pretty close to those.

- Evan

[Nick Sarten]

I tried using the slice header approach, but I ran into more invalid address locations and runtime panics.

After some more searching and research into the reflect package i figured i could actually use reflect.MakeSlice() to create the memory, and then use the .Addr() method to pass it to the C function, instead of malloc'ing a block of memory and trying to convert it to a go type.

Now i'm struggling along with that approach, but already it seems much more straightforward and promising than trying to access a malloc'd block of memory.

- Nick

[roger peppe]

On 12 May 2011 02:37, Nick Sarten <gen.b...@gmail.com> wrote:
> I tried using the slice header approach, but I ran into more invalid address
> locations and runtime panics.
> After some more searching and research into the reflect package i figured i
> could actually use reflect.MakeSlice() to create the memory, and then use
> the .Addr() method to pass it to the C function, instead of malloc'ing a
> block of memory and trying to convert it to a go type.

if you know the type of the slice, you can just use make, and
use the address of the first element.

e.g.

devs := make([]C.cl_device_id, num_devices)
C.clGetContextInfo(c.c_context, C.CL_CONTEXT_DEVICES,
num_devices, &devs[0], &num_devices)

(make sure that num_devices > 0, though, because you can't
take the address of the first element of an empty array)