Is anyone working on a Julia code generator?

[Douglas Bates]

Topic says it all.  http://JuliaLang.org  is the home site for the Julia language.

[mikkelfj]

Not that I am aware of. But I think Julia is one of a few languages where you can take advantage of macros to dynamically create interfaces at runtime, for example based on input from a binary schema.

[Douglas Bates]

I have created a Julia package with the support code at https://github.com/dmbates/FlatBuffers.jl

Right now it can read (most of) the binary object produced from a IDL file.  I am hand-coding the Julia sources from the IDL file.  I haven't yet looked into creating an idl_gen_julia.cpp as I am still tinkering with the representation of table instances as Julia types.

I expect that my code for extracting a string has an off-by-one error in it.  I patterned the code after the JavaScript support code because JavaScript, like Julia, is dynamically typed.

[Douglas Bates]

I have fixed the string problem, I think, and submitted the package to be registered.  Once that comes to pass (usually within a day or two) the package can be added using

Pkg.add("FlatBuffers")

within a Julia session.  If anyone is horribly anxious to try it out before it is registered, use

Pkg.clone("https://github.com/dmbates/FlatBuffers.jl")

[mikkelfj]

Sounds great.

I recommend that you run some buffer verifiers from existing languages bindings on your test output when you get to that part. I'm not sure if the C++ version also checks alignment, but the C version does - I can help with that if needed.

I noticed that you use UTF8String. I'm not sure how strict Julia is on this, but there is a potential performance and compatiblity issue here because users may be tempted to store non-UTF8 strings in FlatBuffers even if it is technically incorrect. An alternative byte vector accessor might be useful? 

[Douglas Bates]

Julia does check a UTF8String  for proper encoding so I guess I could return a Vector{UInt8} instead.

Right now I am having trouble understanding the layout of a union.  It seems that the vtable element is either 0 or an soffset_t to a uint8_t that is the 1-based index into the types in the union.  What is not clear to me is how I get the offset to the instance of that type, once I know the type.

The motivating example for me is the feather storage format, https://github.com/wesm/feather. The IDL file at

https://github.com/wesm/feather/blob/master/cpp/src/feather/metadata.fbs

declares TypeMetadata as a union of four table types: CategoryMetadata, TimestampMetadata, DateMetadata, and TimeMetaData.  Within a Column table, the 'metadata' member is of this union.

If in the vtable for a Column I find that the metadata voffset_t is nonzero, it seems it gives the offset to a uint8_t which indicates the type of table from the union.   How do I find the offset to the table's contents, once I know its type?

I hope this question makes sense.  I can provide a reproducible example if that would help

[mikkelfj]

Julia does check a UTF8String  for proper encoding so I guess I could return a Vector{UInt8} instead.

Or both - javascript takes an optional encoding argument.

Right now I am having trouble understanding the layout of a union.

A union is simple two separate fields in a table, one with an uint8 type, and one with a table reference. The two fields are always neighbours in the vtable, with the type first.

The table reference may be absent if the type is also absent, or if the type is NONE (=0).

So if you known the type has field #2, you can find the table reference by reading field #3, or usually it is the other way around: you substract one from the id of the unions table and find the type vtable entry. You only use the type for detecting NONE and for type casting the reference. The type does not influence how you locate the table reference in the containing table.