Serialization of JanusGraph specific types and search predicates for non-Java based languages

[Florian Hockmann]

JanusGraph uses some types and predicates that are not part of TinkerPop and therefore don't have serializers and deserializers in the different TinkerPop GLVs and language drivers. This means that the following is currently only possible for Java based clients (at least to my best knowledge):

1. Returning those types with Gremlin queries, e.g., g.V(1).outE().next().
2. Adding data to the graph with JanusGraph specific types, like the Geoshape datatype.
3. Using search predicates, e.g., textContains().

Now my question is whether these data types and search predicates should be supported officially by JanusGraph for the different languages or would you prefer them to be developed as third-party projects?

I could help with .NET support, but I wanted to be sure first whether such a project should be started as a PR for JanusGraph or completely independent of it.

My personal opinion is that integrating those (probably relatively small) projects within JanusGraph makes sense as we could then add sections for them to the docs and also ensure that the usage is roughly the same for the different languages. The downside is of course that the build process becomes more complex as it suddenly involves more languages like Python, JavaScript, and C#.

[Jason Plurad]

I'm +1 for JanusGraph housing the client drivers, so users at least wouldn't need to hunt all over for them (unlike storage backends).

[Florian Hockmann]

Since there were no objections to adding client drivers to JanusGraph, I went ahead and created a ticket for the .NET part:

https://github.com/JanusGraph/janusgraph/issues/1059

[Florian Hockmann]

I wonder whether it would actually make more sense to create separate repositories for the different libraries instead of including them in the main repo. This was recently already discussed for Docker images and I think that the advantages of using a separate repo per library are quite similar. Not having to include a non-Java library into JanusGraph's Maven based build and deployment makes the creation of such a library much simpler and it also makes it easier for new contributors to get started as they don't have to set up Maven and Java in addition to the tools they need for the language of the library. It is also easier to manage GitHub issues specific for the library as they don't get included in all the other issues and the reverse is of course also true: .NET (for example) specific issues don't convolute the main issue tracker. It also greatly reduces build times of the libraries as they don't also have to build all of JanusGraph.

I only see two downsides of separate repos: Each library needs to be deployed on its own. So mvn deploy isn't enough any more to deploy everything. Instead someone (ideally more than one person) needs to be responsible for the deployment of each library. The upside of this is of course that libraries could be released more often than JanusGraph itself (although versioning could be complicated in that case). The other disadvantage I see is that testing of the libraries for changes in JanusGraph itself is harder to automate as it needs access to SNAPSHOT builds of JanusGraph Server. We could solve this by publishing nightly builds of JanusGraph Docker images that could then be consumed by the different libraries for testing.

Does that make sense to everyone?

[Jason Plurad]

I think having the client drivers separate from the core code is a decent idea. The JanusGraph build is already complicated enough currently to support the breadth of the storage and indexing backends, so I'd be in favor of a separate repo for client libraries.

Florian, from your experience on TinkerPop, was it worth the effort to get the .NET driver build integrated with the core repo?

[Florian Hockmann]

I'm actually not sure whether it was worth the effort. On the one hand it is really nice that every new step added to Gremlin will automatically also be added to Gremlin.Net but on the other hand I still don't really like the added complexity of Maven for a .NET library, e.g., contributors need Maven to run the tests as Maven sets up the Gremlin Server.
However, since JanusGraph doesn't need code generation for its libraries, the biggest benefit of integrating them into the main repo doesn't apply for JanusGraph.

I'd say the advantages of separate repos outweigh its disadvantages for JanusGraph and we can still integrate them in the main repo if we decide at any point in the future that it was a mistake to put them in separate repos.

[Florian Hockmann]

It is has been 8 days since I proposed to use separate repos for the libraries and nobody disagreed. So, it seems that we have (a somewhat lazy) consensus on this issue.

Could somebody - probably from the TSC - then please create a repo for the .NET library so I can retract my PR against the main repo and instead create it again for that repo? I propose JanusGraph.Net.Extensions as the name for the repo as it's currently the name of the library but I would of course also be OK with another name.

Or do we need a formal VOTE for this decision?

[Jason Plurad]

I'd propose janusgraph-dotnet as the repo name, keeping the name shorter while staying aligned with gremlin-dotnet. I'll start a VOTE thread.