Cannot read serialized file (slice bounds out of range)

[Stepan Bujnak]

I've got following flatbuffers schema:

namespace map_fbs;

table Entry {

  key: string;

  value: [ubyte];

}

table Map {

  Entries: [Entry];

}

root_type Map;

I've serialized and then unserialized many files (>10000) and then I've come across a file that I can't unserialize because calling Map.EntriesLength() panics with runtime error: slice bounds out of range in golang. What is the most likely cause of this problem? Could it be problem with the serialization/deserialization is it simply damaged file? Is there any way to detect damaged file? Thank you.

[mikkelfj]

It sounds like the buffer is invalid which doesn't explain how that happened.

If you used goroutines or other concurrency there is a risk you have used the same builder object to construct two different buffers. It might also be that the file was truncated during file write.

For additinal verification of the buffer, you can use the C++ generated verifier, or the C verifier from flatcc. I am most familiar with the latter: https://github.com/dvidelabs/flatcc#reporting-bugs

The example project will give a hex dump and show an error message. You can also enable further tracing flags in verifier and have a assert on error so you can use lldb or a similar debugger to find out more: https://github.com/dvidelabs/flatcc#debugging-a-buffer and the binary format might also be helpful when investigating: https://github.com/dvidelabs/flatcc/blob/master/doc/binary-format.md

[mikkelfj]

I just realized the bugreport example didn't actually print the error so I fixed that and added it to samples: https://github.com/dvidelabs/flatcc/tree/master/samples/bugreport

[Wouter van Oortmerssen]

Since this schema is so simple, if you can send a hex-dump of the first 50 or so bytes of that file, it is probably easy to see what is wrong.

It should look like:

uint32_t offset_to_root

<alignment>

vtable:

uint16_t size_of_vtable(6), size_of_Map, offset_to_Entries(4)

root:

uint32_t offset_to_vtable

uint32_t offset_to_Entries // This value is is likely corrupt.

--
You received this message because you are subscribed to the Google Groups "FlatBuffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.