Read data while creating, before root element is written with Finish()

[Martin Ankerl]

I am creating lots a tree structure with flatbuffers, and would like to have read access to the data that I have already created before I finish the whole message with Finish(). 

I know there is GetCurrentBufferPointer, but is it possible to access the already created objects somehow?

Martin

[Dmitry Chichkov]

I'm also interested in this functionality.  I'm creating multiple obects, and would like to have read access to objects that I've already created! 
Without having to call Finish on the whole tree!  Alternatively, I have to store the copy of the data, if I need it again, and this is inefficient.

For example:

  flatbuffers::Offset<MyObject> offset = CreateMyObject(builder, 32, 64);
 
It'd be really nice to be able to access the created object fields somehow. Along the lines:

        auto obj = flatbuffers::GetObject<MyObject>(b.GetCurrentBufferPointer());
        // use obj->field1()
or:
        auto obj = flatbuffers::GetObject(builder, offset);
        // use obj->field1()

Or similar...
       
Dmitry

[Martin Ankerl]

If I am not mistaken, This should do the trick:

template<typename T> const T *GetObject(flatbuffers::FlatBufferBuilder& fbb, flatbuffers::Offset<T> offset) {

    return reinterpret_cast<T *>(fbb.GetCurrentBufferPointer() + fbb.GetSize() - offset.o);

}

This seems to work for me, but I am not sure if this works in all cases. Also one has to be aware that the pointers will become invalid when adding new objects and the buffer reallocates.

It might be a good idea to add this to flatbuffers code, it took me a while to figure out how to use the offset

Martin

[Dmitry Chichkov]

That doesn't sound very safe - reinterpret-cast, with the object underneath that can disappear at any time!  I hope no one is writing code like that. 

As a note of caution. I've wasted 2 days writing the code for FlatBuffers and that all went to trash, because FlatBuffers forced me to solutions like the above.
I've switched it to capnproto, and it is much much smoother ride so far.  In the moment, the fact that FlatBuffers was out there was of negative value to me, unfortunately...

Dmitry

[Maxim Zaks]

The C++ API is meant to build up a buffer. When you create a table you are laying out bytes in a contigues binary array. You are not creating objects in an object oriented sense.

The builder let's you practically put all the data that you have stored else where into an array of bytes. 

So in this case reading while writing defeats the purpose, because the purpose is to serialise given data as quickly as possible with minimum amount transient memory.

I am working on FlatBuffersSwift implementation where the code generator generates real classes which can be filled and serialised to a byteArray.

Here is an example for building up a complex cyclical graph, and than serialising and deserialising it.

https://github.com/mzaks/FriendsPlaygroundFB/blob/master/FriendsPlayground.playground/Contents.swift

Here is the schema:

https://github.com/mzaks/FriendsPlaygroundFB/blob/master/FriendsPlayground.playground/Resources/friends.fbs

And here is the generated source file

https://github.com/mzaks/FriendsPlaygroundFB/blob/master/FriendsPlayground.playground/Sources/Friends.swift

As you can see I have a class Friend which is just a normal class I can instantiate it as if it would be a simple class and than call toByteArray() for serialisation.

This is very convenient API, but it is not as fast because the created objects are transient memory from serialisation stand point.

So if you want to serialise data and send it over the wire or save it to disk, the code generated by flatc is perfect.

If you want generated Table classes to be part of your application runtime than you need to generate different API, similar to what I do with FlatBuffersSwift.

PS: @Dmitry Chichkov, reading while writing has nothing to do with FlatBuffers as the serialisation format. It is more about API, if it is designed to do such thing.