flatbuffers and flatcc

[p...@berkeley.edu]

Dear all,

I'm working on a distributed task execution system and would like to use flatbuffers for serializing control messages.

Flatcc is really attractive for the project because it is in C. I'm wondering what the plans for its maintenance are and if I will be able to depend on it in the future. Are there more up to date plans than https://github.com/dvidelabs/flatcc/issues/1 for merging it into flatbuffers? If this doesn't happen, is it reasonable to assume that flatcc will be maintained in the future?

Maybe the author of flatcc can comment on that and it would also be great to see who else is using flatcc right now and potentially helping to maintain it in the future.

Thanks!

[mikkelfj]

Hi, I am the author of flatcc.

I believe the current status is reasonably up to date:

Regarding  https://github.com/dvidelabs/flatcc/issues/1 the only new development is that another recently user asked about how the main flatbuffers project would accept code generation for some less popular language - consensus were that this wasn't the right way to go about it. During that discussion Wouter and I further explored the idea of a new binary schema better suited for code generation for independent implementations. There are no active plans for this, but is likely that if it ever happens, flatcc will port its code generator to be based on this format, but it would be a major effort both in designing the schema, and porting code generators. This could also lead to flatcc supporting more languages in case flatc is not desirable for some reason, e.g. due to size or use of C++.

You may also be interested in my answer to a related question regarding flatcc compatibility with flatbuffers:

https://github.com/dvidelabs/flatcc/issues/15

Regarding future maintenance it is an active design choice that flatcc should be able to be usable far out in into the future without the need for significant maintainance.

Bugs needs to be fixed as they are discovered, and new platforms needs to be supported as users appears. This is very much driven by active users and these changes are imported into flatcc as deemed relevant and I don't see this being discontinued as long as users use the project and contribute.

I expect the flatbuffers format to be quite stable and the flatcc implemention is quite feature complete. Because of this, and because flatcc has no external dependencies, there are not a lot of future concerns related to maintainence. Ideally, a 3+ year old version should continue to work if it works today on a given platform, or worst case, the CMake main file must be updated by the user to accommodate changes in compiler warnings etc. For example, you can safely continue to use the older v0.3.1 if you are on little endian linux, and don't need recent bug fixes to JSON parsing. With version 0.4.0 big endian was finally supported as well, which was the last major feature missing.

Users of less common platforms should be able to manage their own porting without support from the flatcc support, although quality patches will be accepted. This is because the necessary changes are typically isolated the the flatcc/portable library, the CMake build files, and in extreme cases the need for custom runtime allocation in the builder - which has a pluggable interface (some users have been looking into this).

New complex developments in the flatcc schema is not guaranteed to be supported, but that shouldn't affect interoperability for the core flatbuffer features. For example, the schema compiler understands the new RPC syntax, but no code is being generated for grpc like flatc does for C++ and Go. The main reason for this is that grpc is an insanely huge project in its current form that goes against the idea of small compact independent runtime.

The main problem I see with flatcc being separate from flatbuffers (flatc) is that you might not get support from the same package managers as the main project provides - but given that you can pull and build the entire project in a few seconds, it is not a major obstacle, and flatcc is being used where the main flatbuffers project is not supported.

[p...@berkeley.edu]

Hey mikkelfj,

thanks a lot for your detailed answer! It's unfortunate that unifying the compiler seems to be a hard problem and I can see that having the whole toolchain in C can be advantageous. I'm glad that compatibility with flatc seems to be great and the specifications are stable, that covers most of my concerns. I'm also glad that you want to keep flatcc compact.

I have been working some more with flatcc now and really like it so far. Thanks for your work! I wish there was some more example code or documentation on how to get things working concretely (https://github.com/dvidelabs/flatcc/blob/master/samples/monster/monster.c and https://github.com/dvidelabs/flatcc/tree/master/test/monster_test are great though), but I guess that will get better if more projects start using flatcc.

-- Philipp.

[mikkelfj]

Thank you for the kind words.

It would be good with user contributed examples - it would explain things from a different perspective.

Mikkel

[Wouter van Oortmerssen]

Yes, Mikkel has done a great job with his implementation so far, so I wouldn't have too many doubts about it being not part of the main project.

That said, I'd welcome it being merged in some way into the main project, but that does mean doing some things the way the rest of the FlatBuffers project works, e.g. generating code as part of flatc etc.

--
You received this message because you are subscribed to the Google Groups "FlatBuffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.