Hi all,
How can I write a Cython function that takes a byte string object (a
normal byte string, a bytearray, or another object that follows the
buffer protocol [1]) as a typed memoryview [2]? I'm using Python 2.7.
According to the Unicode and Passing Strings Cython tutorial page [3],
the following should work:
cpdef object printbuf(unsigned char[:] buf):
chars = [chr(x) for x in buf]
print repr(''.join(chars))
Unfortunately it doesn't work for normal byte strings and other
read-only buffer objects:
$ python -c 'import test; test.printbuf("test\0ing")'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "test.pyx", line 1, in test.printbuf (test.c:1417)
File "stringsource", line 614, in
View.MemoryView.memoryview_cwrapper (test.c:6795)
File "stringsource", line 321, in
View.MemoryView.memoryview.__cinit__ (test.c:3341)
BufferError: Object is not writable.
But it does work for bytearrays and other writable buffers:
$ python -c 'import test; test.printbuf(bytearray("test\0ing"))'
'test\x00ing'
Looking at the generated C code, Cython is always passing the
`PyBUF_WRITABLE` flag to `PyObject_GetBuffer()`, which explains the
exception for read-only buffer objects.
I can access read-only buffers via PyObject_GetBuffer():
from cpython.buffer cimport \
PyBUF_SIMPLE, PyBUF_WRITABLE, \
PyObject_CheckBuffer, PyObject_GetBuffer, PyBuffer_Release
cpdef object printbuf(object buf):
if not PyObject_CheckBuffer(buf):
raise TypeError("argument must follow the buffer protocol")
cdef Py_buffer view
PyObject_GetBuffer(buf, &view, PyBUF_SIMPLE)
try:
chars = [chr((<unsigned char *>view.buf)[i])
for i in range(view.len)]
print repr(''.join(chars))
finally:
PyBuffer_Release(&view)
It's not as convenient, but it at least behaves as desired:
$ python -c 'import test; test.printbuf(bytearray("test\0ing"))'
'test\x00ing'
$ python -c 'import test; test.printbuf("test\0ing")'
'test\x00ing'
Am I doing something wrong, or does Cython not support coercing
read-only buffer objects (such as normal strings) into typed memoryview
objects?
[1]
https://docs.python.org/2/c-api/buffer.html
[2]
http://docs.cython.org/src/userguide/memoryviews.html
[3]
http://docs.cython.org/src/tutorial/strings.html#accepting-strings-from-python-code
Thanks,
Richard