typed memoryviews: support for read-only buffer objects?

[Richard Hansen]

Hi all,

How can I write a Cython function that takes a byte string object (a
normal byte string, a bytearray, or another object that follows the
buffer protocol [1]) as a typed memoryview [2]? I'm using Python 2.7.

According to the Unicode and Passing Strings Cython tutorial page [3],
the following should work:

cpdef object printbuf(unsigned char[:] buf):
chars = [chr(x) for x in buf]
print repr(''.join(chars))

Unfortunately it doesn't work for normal byte strings and other
read-only buffer objects:

$ python -c 'import test; test.printbuf("test\0ing")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "test.pyx", line 1, in test.printbuf (test.c:1417)
File "stringsource", line 614, in
View.MemoryView.memoryview_cwrapper (test.c:6795)
File "stringsource", line 321, in
View.MemoryView.memoryview.__cinit__ (test.c:3341)
BufferError: Object is not writable.

But it does work for bytearrays and other writable buffers:

$ python -c 'import test; test.printbuf(bytearray("test\0ing"))'
'test\x00ing'

Looking at the generated C code, Cython is always passing the
`PyBUF_WRITABLE` flag to `PyObject_GetBuffer()`, which explains the
exception for read-only buffer objects.

I can access read-only buffers via PyObject_GetBuffer():

from cpython.buffer cimport \
PyBUF_SIMPLE, PyBUF_WRITABLE, \
PyObject_CheckBuffer, PyObject_GetBuffer, PyBuffer_Release

cpdef object printbuf(object buf):
if not PyObject_CheckBuffer(buf):
raise TypeError("argument must follow the buffer protocol")
cdef Py_buffer view
PyObject_GetBuffer(buf, &view, PyBUF_SIMPLE)
try:
chars = [chr((<unsigned char *>view.buf)[i])
for i in range(view.len)]
print repr(''.join(chars))
finally:
PyBuffer_Release(&view)

It's not as convenient, but it at least behaves as desired:

$ python -c 'import test; test.printbuf(bytearray("test\0ing"))'
'test\x00ing'
$ python -c 'import test; test.printbuf("test\0ing")'
'test\x00ing'

Am I doing something wrong, or does Cython not support coercing
read-only buffer objects (such as normal strings) into typed memoryview
objects?

[1] https://docs.python.org/2/c-api/buffer.html
[2] http://docs.cython.org/src/userguide/memoryviews.html
[3]
http://docs.cython.org/src/tutorial/strings.html#accepting-strings-from-python-code

Thanks,
Richard

[Stefan Behnel]

This has been requested a couple of times already. I wonder if this
couldn't be achieved by declaring the item type "const", e.g.

cdef const int[:] = b'...'

This is obviously not implemented yet, but would it provide a suitable
syntax for this feature, or is there anything I'm missing?

Stefan

[Sturla Molden]

On 26/07/15 17:40, Stefan Behnel wrote:

> This has been requested a couple of times already. I wonder if this
> couldn't be achieved by declaring the item type "const", e.g.
>
> cdef const int[:] = b'...'
>
> This is obviously not implemented yet, but would it provide a suitable
> syntax for this feature, or is there anything I'm missing?

The const modifier should only affect the generated C code, it should
not require a readonly buffer. It should be legal to pass a writable
buffer to a typed memoryview declared cdef const int[:]. OTOH, readonly
buffer should require the const modifier in the memoryview declaration.


Sturla

[Ecolss Logan]

Has this "const" feature been implemented now?

I still cannot seem to create a typed memoryview on a read-only buffers yet ...

在 2015年7月27日星期一 UTC+8上午2:49:13，Sturla Molden写道：