"Correct" way of accessing pointer to numpy data

[Henry Gomersall]

Can someone advise me what the current correct way of accessing numpy
pointers is?

My current code uses PyArray_Data(array), which seems to work fine but I
understand is now deprecated. Memoryview seems to be the preferred
option. Can someone confirm this? Also, are memoryviews just pointers?

Interestingly, I can compile the code just fine using gcc and mingw
(under windows), but when I try to use the Windows SDK compiler, I get a
compile fail - presumably because the relevant macros are not being
imported as NPY_NO_DEPRECATED_API is not set correctly (should that be
set or not set to allow the code to compile?).

Cheers,

Henry

[Henry Gomersall]

On 19/09/13 13:34, Henry Gomersall wrote:
> Also, are memoryviews just pointers?

Sorry, I'll rephrase that. Can memoryviews be treated like pointers?

Henry

[Chris Barker - NOAA Federal]

On Sep 19, 2013, at 5:34 AM, Henry Gomersall <he...@cantab.net> wrote:

Can someone advise me what the current correct way of accessing numpy pointers is?

My current code uses PyArray_Data(array), which seems to work fine but I understand is now deprecated.

According to Sturla:

Py_Array_DATA

Is the new, non-depricated method ( is that the same ?). What's depricated is accessing the .data field of the  ndarray struct.

But using a memory view is a more flexible option, as it can be built from any PEP 3118 buffer.

Memoryview seems to be the preferred option. Can someone confirm this? Also, are memoryviews just pointers?

&a_memview[0] is the pointer to the beginning of the buffer, so you can use that. But why not take advantage of memoryview features if you have one?

when I try to use the Windows SDK compiler, I get a compile fail - presumably because the relevant macros are not being imported as NPY_NO_DEPRECATED_API is not set correctly (should that be set or not set to allow the code to compile?).

What versions are you using? This works for me, though with warnings...

CHB

Cheers,

Henry

--

--- You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Henry Gomersall]

On 19/09/13 15:45, Chris Barker - NOAA Federal wrote:

On Sep 19, 2013, at 5:34 AM, Henry Gomersall <he...@cantab.net> wrote:

Can someone advise me what the current correct way of accessing numpy pointers is?

My current code uses PyArray_Data(array), which seems to work fine but I understand is now deprecated.

According to Sturla:

Py_Array_DATA

Is this a typo? or is Py_Array_DATA different to PyArray_DATA?

Is the new, non-depricated method ( is that the same ?). What's depricated is accessing the .data field of the  ndarray struct.

But using a memory view is a more flexible option, as it can be built from any PEP 3118 buffer.

Memoryview seems to be the preferred option. Can someone confirm this? Also, are memoryviews just pointers?

&a_memview[0] is the pointer to the beginning of the buffer, so you can use that. But why not take advantage of memoryview features if you have one?

I don't think I really need memoryview features. I'm basically just doing 2 things: (i) passing pointers to data to a C function and (ii) checking the alignment of an array in memory (from the pointer).

when I try to use the Windows SDK compiler, I get a compile fail - presumably because the relevant macros are not being imported as NPY_NO_DEPRECATED_API is not set correctly (should that be set or not set to allow the code to compile?).

What versions are you using? This works for me, though with warnings...

This is with "Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1":
http://www.microsoft.com/en-gb/download/details.aspx?id=3138

Python 2.7.1, Cython 19.1 and numpy 1.7.1.

This is really my first foray into building on windows using MS's compiler. I've had lots of joy with mingw, but there is some demand for a 64-bit build (and I now have a 64-bit installation of Windows).

Are there some magic defines I need to pass to the compiler?

If you're interested, the code is here: https://github.com/hgomersall/pyFFTW (I would certainly appreciate some insights!)

Cheers,

Henry

[Robert Kern]

On 2013-09-19 16:16, Henry Gomersall wrote:
> On 19/09/13 15:45, Chris Barker - NOAA Federal wrote:
>> On Sep 19, 2013, at 5:34 AM, Henry Gomersall <he...@cantab.net

>> <mailto:he...@cantab.net>> wrote:
>>
>>> Can someone advise me what the current correct way of accessing numpy
>>> pointers is?
>>>
>>> My current code uses PyArray_Data(array), which seems to work fine but I
>>> understand is now deprecated.
>>
>> According to Sturla:
>>> Py_Array_DATA
>>
> Is this a typo? or is Py_Array_DATA different to PyArray_DATA?

Typo. The correct (and never-been-deprecated) macro is PyArray_DATA().

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Henry Gomersall]

On 19/09/13 16:21, Robert Kern wrote:
> On 2013-09-19 16:16, Henry Gomersall wrote:
>> On 19/09/13 15:45, Chris Barker - NOAA Federal wrote:
>>> On Sep 19, 2013, at 5:34 AM, Henry Gomersall <he...@cantab.net
>>> <mailto:he...@cantab.net>> wrote:
>>>
>>>> Can someone advise me what the current correct way of accessing numpy
>>>> pointers is?
>>>>
>>>> My current code uses PyArray_Data(array), which seems to work fine
>>>> but I
>>>> understand is now deprecated.
>>>
>>> According to Sturla:
>>>> Py_Array_DATA
>>>
>> Is this a typo? or is Py_Array_DATA different to PyArray_DATA?
>
> Typo. The correct (and never-been-deprecated) macro is PyArray_DATA().

hmmm according to ndarraytypes.h...

/* These macros are deprecated as of NumPy 1.7. */
#define PyArray_NDIM(obj) (((PyArrayObject_fields *)(obj))->nd)
#define PyArray_BYTES(obj) (((PyArrayObject_fields *)(obj))->data)
#define PyArray_DATA(obj) ((void *)((PyArrayObject_fields *)(obj))->data)
#define PyArray_DIMS(obj) (((PyArrayObject_fields *)(obj))->dimensions)
etc
etc

Henry

[Henry Gomersall]

Except it's also defined above inside
#if (defined(NPY_NO_DEPRECATED_API) && (NPY_API_VERSION <=
NPY_NO_DEPRECATED_API))

I'm thoroughly lost now. I need to see the post-pre-processed output,
and also all the macro defines. I've basically no idea what the build is
doing to make it fail.

Cheers,

Henry

[Robert Kern]

The macro forms have been deprecated in favor of the inline function
definitions. At the level of the C sources, nothing is changing. You still will
write code using PyArray_DATA() going into the future. However, binaries
compiled against older versions of numpy will eventually stop working with newer
builds of numpy.

[Chris Barker]

On Thu, Sep 19, 2013 at 8:21 AM, Robert Kern Typo. The correct (and never-been-deprecated) macro is PyArray_DATA().

Sorry for the confusion -- I was on a phone and couldn't easily look it up.

Henry -- if you you care about being able to recieve actual nmy arrays, then it sounds like you should just use PyArray_DATA(). Memoryviews give you:

1) they can take other PEP 3118 compliant objects

2) they let you index into them, etc.

But I dont think they give you anything if you just want the pointer.

By the way, I'm not sure what the Cython Wiki is trashed with spam, but there was a page here:

http://wiki.cython.org/tutorials/NumpyPointerToC

that it would be nice to update.

NOTE: we clearly need a new or better Wiki solution -- which will take someone doing some work -- but in the meantime, is it possible to roll it back and make it read-only?

-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

[Robert Bradshaw]

On Thu, Sep 19, 2013 at 9:26 AM, Chris Barker <chris....@noaa.gov> wrote:
> By the way, I'm not sure what the Cython Wiki is trashed with spam, but
> there was a page here:
>
> http://wiki.cython.org/tutorials/NumpyPointerToC
>
> that it would be nice to update.

I've reverted this.

> NOTE: we clearly need a new or better Wiki solution -- which will take
> someone doing some work -- but in the meantime, is it possible to roll it
> back and make it read-only?

I went through and deleted several thousand pages of spam and reverted
everything linked to from the main page at the beginning of the month.
I also made it "read-only" by choosing a nonsensical capacha (if you
want it, let me know) until we find a long-term solution.

- Robert