Length Prefixing Thoughts

[Sam Paioletti]

I was working on a project that required me to use the currently accepted method of length prefixing flatbuffers to chain them. (i.e. len>fb>len>fb)

It occurred to me that we might be able to do this with an optional field in the root table with an attribute:

table Monster {

  length:uint(length);

  pos:Vec3;

  color:Color = Blue;

  test:Any;

}

root_type Monster;

This way in our GetRootAsMonster generated code we look for this field and a) if not set then use entire buffer b) if set use as length for buffer (optionally return next index or something to flag more to follow)

I'm sure there maybe some catch to this, but if this was possible it would remove the incompatibility between length prefixed/normal flatbuffers and wouldn't require much overhead. Tables without the length attribute would still assume the entire buffer was valid.

Thoughts?

[mikkelfj]

This would work for a custom solution, but not in general because it is places constraints on the schema. Also, you cannot know the length before writing the buffer so you have to rely on mutating the buffer subsequently. While this is possible to do, it is not necessarily trivially supported in all languages. Also, it is not the fastest solution if you need to skip through a set of buffers until you find one with the desired buffer identifier.

If you need to be able to distinguish one buffer type over the other, I suggest setting the most signficant bit of the buffer identifier instead. If the first 4 bytes is a length, it would be shorter than 2^31.

[Wouter van Oortmerssen]

I think the current solution of length-prefixed buffers is nicer. If we work towards supporting it in all languages out of the box like we do in C++, it can even be a bit of a "standard".

--
You received this message because you are subscribed to the Google Groups "FlatBuffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.