Parquet I/O Format for TinkerPop

[Taylor Riggan]

I'm curious if anyone here has researched or developed a Parquet I/O Format (https://tinkerpop.apache.org/docs/3.4.10/reference/#_input_output_formats) for TinkerPop?  Most enterprises have begun to develop data lakes with Parquet as the standard data/storage format.  Having the ability to export/import to/from Parquet from/to a TinkerPop-based graph database might be useful.

Open to any feedback, thoughts, and/or experiences related to Parquet and graph in general.

Cheers,

Taylor

[Joshua Shinavier]

I haven't!

:-)

With the above disclaimer that I have not looked into Parquet specifically, and do not have a clear picture of how challenging Parquet's idiosyncrasies (e.g. its preference for column-friendly flatness, and the lack of a primitive string type) may be to deal with, I agree that support for Parquet in TinkerPop would be interesting, and probably similar to support for Thrift, Protobuf, and Avro as recently illustrated in the How to Build a Dragon series of presentations.

Josh

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/671a4ba3-3de7-48fe-acdb-74f11ac4b17an%40googlegroups.com.

[Stephen Mallette]

what do you do for string? just coerce to binary i guess? 

we really haven't looked at on disk file formats since we started TinkerPop 3. some fresh ideas in this area may be in order.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAPc0OuseHh_BPp2atoGWuD8gs9iaZA8x7Z_pMeUS3COM0dYR_Q%40mail.gmail.com.