Gremlin 2.0? -- A Graph DSL Explosion

[Marko Rodriguez]

Hello everyone,

So we have an idea for the future of Gremlin/Pipes. Now that Pipes has all the functionality that Gremlin had (given the addition of PipeClosure), it is possible to provide "Gremlin-esque" graph traversing to other JVM languages, especially those that provide closures or lamda-functions. This idea has been made obvious by the transition from ANTLR Gremlin to Groovy Gremlin and the developed of JRuby Pacer.

We think a huge gain for TinkerPop community would be to blow out Gremlin in this fashion.

Gremlin(groovy)
Gremlin(clojure)
Gremlin(jruby)
Gremlin(jython)
Gremlin(rhino)
Gremlin(scala)

...where given that all the functionality is in Pipes, we simply make use of the conventions of each language:
outE (groovy, rhino, scala)
out-e (clojure)
out_e (ruby)

...and we create a naming/style convention that is consistent through all languages to easily allow people to jump between them if they so desire. Moreover, to ensure that the emergent documentation style is consistent across all languages.

Finally, we have a single Gremlin console (and JSR223 implementation) that mimics the functionality brought forth by Mutant/RexsterConsole:

gremlin[groovy]> ?clojure
gremlin[clojure]> (setf x 1+2)
==>3
gremlin[clojure]> ?jruby
gremlin[jruby]> x
==>3

gremlin[jruby]> ?groovy

gremlin[groovy]> g.v(1).outE.filter{it.weight > x}.inV

With this approach, we do 3 radical things:

1. We promote not a language, but a style of graph traversing that is easily emulated in other languages.
- We explore a new model of documentation that must be language agnostic and easy to consume irrespective of language.
2. Gremlin wins over more people as they can now use the "Gremlin style" (the Pipes style) in their native languages (and in their native IDEs with ease).

- We must ensure that all Gremlin language versions are tightly coupled in concept/naming/style/etc.
3. Rexster wins over more people as RexsterConsole becomes a remote JVM communication platform.
- Rexster becomes more of a JVM server with graph engine

We see this as something we could release with Gremlin 2.0. Gremlin is seen a graph-traversal style where any JVM language can take advantage of that style as it simply compiles down to Pipes. Also, we can start small with only ~3 languages to test the waters and to not overwhelm ourselves as we figure out how this is all going to play out.

If all goes well, perhaps in Spring 2012, this all emerges from the TinkerFactory here in NoWhereVille (for where Pavel is the Mayor).

The communities thoughts are more than welcome,
Marko.

http://markorodriguez.com

[Jack Park]

I've started investigating the Scala/Lift platform: Gremlin(scala)
would be a huge win there.

Jack

[Luca Garulli]

Wow! Impressive... I'm sure Gremlin/Pipes will become the de-facto language to work with graphs like XPath is for XML!

Lvc@

[Norbert Tausch]

This would be a great advantage. But it will also be difficult. Scala, for example is no scripting language and therefore more difficult to be integrated in other JSR223 languages. Furthermore, Scala comes with an own powerful collections library. But when using Gremlin/Pipes, we have to use Java iterables. In order to integrate these iterables seemlessly into the common Scala code, the start and end iterables of a pipeline have to be converted. Although this is possible, it seems unnecessary. Another point is, also Blueprints is Java based and therefore the properties map is a Java map. This map doesn't provide the functionality of a Scala map and can't be integrated well into the functional, monad based API of Scala. Therefore the Blueprints map also has to be converted. Otherwise Gremlin will be some sort of API which can be used within Scala but it isn't seemlessly integrated.
I think it would be a great possibility if there is a Gremlin dialect in Scala, but there will be some limitations which must be considered. But also a basic integration in Scala would be great.

Best regards

Norbert Tausch

[Daniel Quest]

Let me weigh in.

I think architecturally this is a good choice.  I only have one concern.  IMHO people need to be able to develop applications in Gremlin/Groovy/Jthon or whatever.  Applications require 1) building custom pipes and 2) scripts that use the pipes.  I view my role in all of this to help spread the good word of graph and graph databases to the molecular biology / bioinformatics community.  I would like to build tools specific to their needs on top of the current tinkerpop stack, and distribute them in such a way that it is no harder to use than the current default standard, Perl.  as far as I can tell JSR223 does not support Perl... so I would like something that scripts as easy as Perl AND has all of the BioPerl functionality AND works in the graphDB space.  This is a HUGE win for my community.  I think the current Gremlin is great and satisfies all these requirements.  I think Marko's proposal may be even better, but we should be careful in not introducing new complexity with the features.  I don't believe most programmers will commonly write scripts in three languages; that feels a lot like the programmers I know who make the most cryptic programs possible just to say they did it.  I think the objective is to grow the gremlin developer community.  Adding Jython (and other languages people already know) would be a huge win for that.  The more user apps exist, the more questions, and the more clean the stack gets which is a great thing from the usability perspective but also from the functionality perspective.

Good idea

Daniel

[Daniel Quest]

I thought I would add some conversations from the past on this issue:

First when we talked about mutant, there is this problem with JSR223 implementations.

Josh Shinavier built Jython 2.5.1 which now comes with JythonScriptEngine. Good news, we have Jython inMutant. Bad news, their implementation is incorrect. Moreover, its the MOST incorrect of all the ScriptEngines I've seen---variables can't go in or out (Jython is a closed system). Its amazing, Clojure, jRuby, and Jython all have bad implementations of JSR 223. They are not using bindings correctly. Rhino, Groovy, AppleScript, and Gremlin are the only languages that are correctly implementing the specification.

Did this ever get cleared up?

Second was this use case with R, to which Marko said, Dude this is dope!

"""

Hi there, The other day I was wondering how easy it would be to use my Blueprints graphdb in R. And here's how easy it is. Hopefully someone will find this useful. :-) Later on, I might do some R bindings for Blueprints, if I have the time. Meanwhile, check rJava documentation to understand how to map your Java code to rJava syntax. library(rJava) .jinit() lib.dir <- "/Replace/With/JAR/Directory" graphdb.path <- "Your/GraphDB/Here" vertex.property.name <- "justAPropertyToTest" .jaddClassPath( paste(lib.dir, "blueprints-core-0.6.jar", sep="/") ) .jaddClassPath( paste(lib.dir, "blueprints-neo4j-graph-0.6.jar", sep="/") ) .jaddClassPath( paste(lib.dir, "neo4j-kernel-1.3.M05.jar", sep="/") ) .jaddClassPath( paste(lib.dir, "neo4j-management-1.3.M05.jar", sep="/") ) .jaddClassPath( paste(lib.dir, "geronimo-jta_1.1_spec-1.1.1.jar", sep="/") ) .jaddClassPath( paste(lib.dir, "neo4j-lucene-index-1.3.M05.jar", sep="/") ) .jaddClassPath( paste(lib.dir, "org.apache.servicemix.bundles.lucene-3.0.1_2.jar", sep="/") ) attach(javaImport("com.tinkerpop.blueprints.pgm.impls.neo4j")) g <- new(Neo4jGraph, graphdb.path) it <- g$getVertices() c <- 0 while (.jrcall(it, "hasNext")) { v <- .jrcall(it, "next") print(v$getProperty(vertex.property.name)) c <- c + 1 if (c > 10) { break } } g$shutdown() """

[Marko Rodriguez]

Hey,

Its amazing, Clojure, jRuby, and Jython all have bad implementations of JSR 223. They are not using bindings correctly. Rhino, Groovy, AppleScript, and Gremlin are the only languages that are correctly implementing the specification.

Did this ever get cleared up?

The Jython guys got back to me and said they fixed their JSR 223 implementation. JRuby and Clojure, not that I know of. 

Marko.

http://markorodriguez.com