Solidity (Ethereum) support

[Jean-Marc Le Roux]

Hello,

I'm working on an open blockchain-powered online voting platform: https://github.com/promethe42/cocorico

Along the road, I noticed the Solidity (the language that compiles down to the Ethereum VM assembly) ABI is full of constraints regarding typing.

I thought it could be very useful to be able to work with FlatBuffers back and forth such ABI, but also between Solidity smart contracts.

I think FlatBuffers schemas would be really neat to automate a type-safe interface between the Solidity blockchain code and the "oracles" code written in JavaScript or other languages.

It would be also very useful to define versionable service-oriented interfaces between Solidity smart contracts.

So I started working on a Solidity port of FlatBuffers:

https://github.com/promethe42/flatbuffers/blob/feature/solidity/solidity/flatbuffers.sol

I was wondering:

• if other people are interested and would like to contribute ?
  
• if it makes sense to do a pull request when it works ?
  
• if the fact that Solidity doesn't support floats as of now was a problem regarding that potential pull request (schemas using floats would issue an error when compiled with flatc for now) ?
  

Thank you!

PS : it's worth noting that Solidity doesn't support bitshift operators just yet. But it's in the works so it should land pretty soon.

[Wouter van Oortmerssen]

Interesting, hadn't heard of this language.

So far FlatBuffers has only received ports for very mainstream languages, so I am not sure what we'd want to do in this case. On the one hand I am a fan of supporting as many languages as possible, on the other hand, once it is in the main repository, it becomes our burden to support it (and make sure it doesn't break).

It is possible to write code generators outside of flatc, since you can write a program in any language that FlatBuffers already supports and reads a binary schema, and then generates code.

As for not supporting floats, yes, and error in the code generator would make sense.

--
You received this message because you are subscribed to the Google Groups "FlatBuffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[mikkelfj]

On Monday, November 21, 2016 at 5:47:03 PM UTC+1, Wouter van Oortmerssen wrote:

It is possible to write code generators outside of flatc, since you can write a program in any language that FlatBuffers already supports and reads a binary schema, and then generates code.

I also have been considering this approach for some less languages.

One problem is that the binary schema is not enough for a really solid code generator with multiple include files and scope handling, and possibly enum resolution and binary search. I do not recall. At some point I considered creating an extension schema that would use the binary schema but add information. This proved complex. So I think one should either use the current binary schema and accept its limitations, or a new independent schema should be developed.

Another approach I have been considering is to generate the schema to Lua tables and then allow lua based code generators as plugins. One does not have to write the code generator in the native language.

And finally, I am also considering a hybrid approach where the binary schema is used to generate wrappers for code already generated for C or C++.

This last approach is probably the simplest and most powerful for less mainstream languages.

 

As for not supporting floats, yes, and error in the code generator would make sense.

On Mon, Nov 21, 2016 at 12:08 AM, Jean-Marc Le Roux <jeanmar...@gmail.com> wrote:

Hello,

I'm working on an open blockchain-powered online voting platform: https://github.com/promethe42/cocorico

Along the road, I noticed the Solidity (the language that compiles down to the Ethereum VM assembly) ABI is full of constraints regarding typing.

I thought it could be very useful to be able to work with FlatBuffers back and forth such ABI, but also between Solidity smart contracts.

I think FlatBuffers schemas would be really neat to automate a type-safe interface between the Solidity blockchain code and the "oracles" code written in JavaScript or other languages.

It would be also very useful to define versionable service-oriented interfaces between Solidity smart contracts.

So I started working on a Solidity port of FlatBuffers:

https://github.com/promethe42/flatbuffers/blob/feature/solidity/solidity/flatbuffers.sol

I was wondering:

• if other people are interested and would like to contribute ?
  
• if it makes sense to do a pull request when it works ?
  
• if the fact that Solidity doesn't support floats as of now was a problem regarding that potential pull request (schemas using floats would issue an error when compiled with flatc for now) ?
  

Thank you!

PS : it's worth noting that Solidity doesn't support bitshift operators just yet. But it's in the works so it should land pretty soon.

--
You received this message because you are subscribed to the Google Groups "FlatBuffers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers...@googlegroups.com.

[Wouter van Oortmerssen]

I think that extending the binary schema to have more complete information is a more flexible approach than integrating a scripting language or a plugin system.

If we can nail down what all exactly is missing, we can simply add it to the binary schema. If for whatever reason this information would bloat the binary schema too much we can also opt for a switch.

To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers+unsubscribe@googlegroups.com.

[mikkelfj]

On Monday, November 21, 2016 at 10:23:40 PM UTC+1, Wouter van Oortmerssen wrote:

I think that extending the binary schema to have more complete information is a more flexible approach than integrating a scripting language or a plugin system.

If we can nail down what all exactly is missing, we can simply add it to the binary schema. If for whatever reason this information would bloat the binary schema too much we can also opt for a switch.

This thread is probably not the right thread for a detailed discussion, but a quick summary of issues I vaguely recall:

 

There are two kinds of information needed: the information to allow exact code generations, and information to make code generation simple. The last part could lead to bloat.

Also, the current schema uses a DAG in a few places which makes efficient JSON translation impossible which is unfortunate for those languages that prefer to bootstrap from JSON, notably a more complex binary schema could be heavy to support in each new language.

Different languages generate code differently, but one requirement I have is that each schema should generate the exact same set of targetfiles regardless of whether the it was included or a root schema.

Identifiers are now fully qualitfied. This is big improvement.

Each identifier need to specify which file it belongs to (I think), and there need to be a list of files and a map to the the symbols it contains. The should be topologically sorted list of structs and tables based on their interdependencies. There should probably be two ordered list for a struct, one for definition order, and one ordered by attribute criteria, or this could be skipped and already decided before generating the binary schema. However, ideally the binary schema is able to print the text again without loss of information.

A special problem is the scenario where root schema file A includes file B and C. When C is used as root, it cannot see content of B, but if used in A, B might accidentally see content of B. Therefore each each schema file has its own view of the visible symbol table. If the code is always generated only for one specific root, the semantics can change so C may see B but then code generation becomes ambiguous.

Scope resolution (as flatcc defines it at least) starts by the scope of the current table unqualified, then relative the active namespace for the given table, and then as a fully qualfiied scope. This require that each symbol has information about these scope contexts.

 

The flatcc code generator the for JSON parser actually emits such scope tables in order to handle enum name lookups correctly, and it isn't as bad as it sounds, but it does prevent certain simplifications.

In addition there may be a need for more detailed alignment hints, to simplify codegenerations, but I am not sure, the current schema already has some information.

[mikkelfj]

when I say struct order list, I mean table field order list. But topo sort refers to structs as well.

Custom attributes should also be supported in a way that makes this feature useful in praxis.

[Wouter van Oortmerssen]

Things like files can be made compact by sharing them among all fields, again as a DAG.

Hard to say without having a sample use case.

--