Numpy record arrays with nested array

[Michael Hogg]

Hi,

I have python code that uses a numpy record array. Once of the records is a 3x1 array of floats, created using:

nodes = numpy.zeros(numNodes,dtype=[('label','|i4'),('coordinates','|f4',(3,))])

Is it possible to do a similar thing in Cython that will give fast indexing? I have found some information on how to do it without a nested array using packed struct, but my attempts to do with a nested array typically give me the error (not verbatim) "dtype does not exist in numpy.pxd"

Any help appreciated.

Mike

[Jake Vanderplas]

Hi Mike,

As far as I know, there's no C type associated with the dtype you use above.  If someone knows better than me, feel free to chime in!

Here's one thing that works to at least get fast cython access to a compatible type.  You don't get indexing within the coordinate array, but this should be a start:

%%cython

import numpy as np

cdef struct Coords:

    float c1

    float c2

    float c3

cdef struct NodeData:

    int label

    Coords coordinates

# hack to get associated numpy data type

cdef NodeData dummy

cdef NodeData[:] dummy_view = <NodeData[:1]> &dummy

NodeData_py = np.asarray(dummy_view).dtype

def func():

    cdef int i

    x = np.zeros(4, dtype=NodeData_py)

    cdef NodeData[::1] xptr = x

    for i in range(4):

        xptr[i].label = i

        xptr[i].coordinates.c1 = i + 0.5

    y = np.zeros(4, dtype=[('label','|i4'),('coordinates','|f4',(3,))])

    y[:] = x

    print y

calling func() should execute without an error, and the data in y should be the same as the data we put into x.  The problem is that you can't index the coordinates using this approach... If you really want to be able to index each coordinate within a node, you could probably cast a raw C array of nodes to a type that looks like

cdef struct NodeType2:

    cdef int label

    cdef float[3] coordinates

I haven't tried this though.

If you really need to index the coordinates within cython and also be able to access them in numpy, I would consider using two arrays: one for the node labels, and one for the coordinates.  This is the "struct of arrays" approach, rather than the "array of structs" approach, and might actually result in faster code in some situations.

I hope that helps,

   Jake

 

--
 
---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Jake Vanderplas]

Oops, that should be `float coordinates[3]`