'jep.PyJArray' does not have the buffer interface

[M]

Hello,
I'm trying to pass a Byte[] array as a parameter into a Python script where io.BytesIO function is reading those bytes:
jep.set("input", bytes);
jep.eval(...);

I'm getting this error:

'jep.PyJArray' does not have the buffer interface

Is there another data type I could use so that io.BytesIO would work?
Thanks,

M

[Ben Steffensmeier]

M,

I am not very familiar with the python buffer interface but it looks like something that numpy objects would implement. If that is the case then using NDArray instead of a Byte[] should result in a python object that is compatible.

I don't think there is anyway we could implement the buffer interface for a Byte[] because JNI does not expose the underlying memory structure for java objects. It should be possible to alter jep to implement the buffer interface for a byte[] or other primitive arrays although I am not sure it would offer advantage over using the numpy types.

Ben

[SemanticBeeng]

Hi Ben,

Getting  " <class 'TypeError'>: a bytes-like object is required, not 'jep.PyJArray'"

When calling a Python function with a Scala object containing Array[Array[Byte]].

The receiver is an apache arrow method evidently requiring an array of bytes.

Any insights why this wrapping if happening for the nested array?

Many thanks

Nick

[SemanticBeeng]

Also trying to use DirectNDArray since I could really use the sharing of memory for the byte array.

Found this useful "   # Getting the same object from a java method triggers the automatic conversion to an ndarray."

https://github.com/ninia/jep/blob/2af843dc55b5d8420c520153081acd041ba5022b/tests/test_numpy.py#L120-L121

Marshalling seems to work.

But how do I actually get the byte array in Python? Ideally without copying?

Buffer array() is not allowed for direct buffers.

https://stackoverflow.com/questions/679298/gets-byte-array-from-a-bytebuffer-in-java

http://www.pandurangpatil.com/2014/04/java-retreive-bytearray-from-direct.html

There are examples to copy data out but this kind of defeats the purpose of sharing memory.

Also, again, how to do this to get Python bytes?

Thanks in advance.

Nick

[SemanticBeeng]

UPDATE: about "Is there are way to get the nio.Buffer bytes to Python?"

    bytes_data = bytes(buffer.remaining())

    buffer.get(bytes_data, 0, buffer.remaining())

gives

" jep.JepException: <class 'TypeError'>: Error converting parameter 1: Expected [B but received a bytes"

[Nathan Jensen]

bytes/bytes_data is a Python object.  buffer is a Java object.  You can't pass a Python object to a Java method unless it's a supported conversion, which bytes is not.  Please read this documentation: https://github.com/ninia/jep/wiki/How-Jep-Works#objects

You should use the numpy support to get a byte array.  Otherwise you could try using buffer.get() in a loop.

--
You received this message because you are subscribed to the Google Groups "Jep Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jep-project+unsubscribe@googlegroups.com.
To post to this group, send email to jep-p...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jep-project/f88b4519-b928-4bd8-b35d-a08f5fa9d1a5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[SemanticBeeng]

Used get() in a loop and works but it is not a production solution, of course.

Solved as per https://github.com/ninia/jep/issues/139 - thanks again, Nate.

To unsubscribe from this group and stop receiving emails from it, send an email to jep-project...@googlegroups.com.

To post to this group, send email to jep-p...@googlegroups.com.