Strange behavior depending on Java ByteBuffer provenence...

[paul Bandler]

I have encountered some odd behaviour when accessing a flat-buffer that has been saved to a file, read-back and decoded.  The format of the file is:

 

<length-integer><flat-buffer bytes as returned from FlatBufferBuilder#sizedByteArray>

 

If I read back into directly allocated (i.e. off-heap) ByteBuffers all is fine, but when using heap-allocated byte buffers problems occur when reading a string.  The difference is attributable to the different path that the Table#_string method takes depending on whether ByteBuffer#hasArray returns true or false.

 

The problem can be reproduced from the following trivial flat-buffer schema (also attached) and the unit tests.

 

namespace flatbuffers.generated;

table MyTable {
    myString:string;
}

root_type MyTable;

The 4 test cases illustrate:

- decoding the binary byte[] as directly retrieved from FlatBufferBuilder and then re-wrapped in a ByteBuffer works fine of course

- decoding the serialised form (<length><flat buffer byte[]> from a directly allocated ByteBuffer works

- decoding the serialised form (<length><flat buffer byte[]> from a heap allocated ByteBuffer fails

- decoding the serialised form (<length><flat buffer byte[]> from a byte[] wrapped ByteBuffer fails

If the Table#_string method is changed to always take the path as per no support for hasArray, then all work fine.  This suggests that it's something to do with the endian-ness of the buffer, maybe at the point that the string's length of obtained, but I am unable to determine what exactly as the buffer properties look ok when stepping thru' the code with the debugger.

Any suggestions as to what I'm doing wrong or the cause would be most appreciated

 

 

[Wouter van Oortmerssen]

Can you define what "problems occur when reading a string" means? What is the exact error, or what data are you getting back?

--
You received this message because you are subscribed to the Google Groups "FlatBuffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[paul Bandler]

The string returned, isn't the string that was written in the flat-buffer but something that is clearly starting with non-ascii and possibly the wrong length.  When the JUnit test code I attached is run the failing cases show that the reported String is rendered as what appears to be an extended ascii character for a dot or diamond character.

[Wouter van Oortmerssen]

Hmm.. the test included in FlatBuffers tries strings both with read-only and regular ByteBuffers.

Can you provide some test code that shows the failure?

[paul Bandler]

My first posting on this topic includes 2 attachments, a trivial flat buffer schema and Junit test style class that illustrates the problem. At least it does when run on a the windows jvm that I'm using which I think is Oracle 1.7 (not at the office now to check)

[Patrick Julien]

Reported ticket

https://github.com/google/flatbuffers/issues/188

PR is here

https://github.com/google/flatbuffers/pull/189