gremlin-x

372 views
Skip to first unread message

Marko Rodriguez

unread,
Nov 30, 2016, 9:06:35 AM11/30/16
to d...@tinkerpop.apache.org, gremli...@googlegroups.com
Hello,

I think TinkerPop has too many ways of doing things. For instance:

1. Users can use graph traversal or interact with vertex/edge/etc. Java objects directly.
2. Users can interact with the graph system embedded (via the Graph object) or remotely (via RemoteConnection).
3. Users can submit Gremlin string scripts or submit Gremlin bytecode to GremlinServer (along with other REST/etc. type capabilities possible).
4. Users can use GraphML, GraphSON, or Gryo for serialization.
5. Users can use Gremlin-Groovy, Gremlin-Java, Gremlin-Python, Gremlin-Scala, etc.
6. Users can use Titan, Neo4j, OrientDB, etc.

I think we should create a module called gremlin-x/ which is the most simplified, concise, recommended way to use TinkerPop. In correspondence to the itemization above:

1. Users can only use graph traversal. All returned elements are ReferenceElements.
2. Users can only interact with the graph remotely (where remote may just be a localhost connection — in-memory proxy (e.g. for TinkerGraph)).
3. Users can only submit Gremlin bytecode to GremlinServer (thus, only RemoteConnection can be used). No more security issues, no more uninterruptible scripts, etc.
4. Users can only use GraphSON as its language agnostic.
5. Users can use any Gremlin language variant as long is generates Gremlin bytecode.
6. Users can use any graph system provider as long as it has a RemoteConnection exposed (where the provider can simply leverage GremlinServer).

Finally, we should create a separate documentation called gremlin-x.asciidoc which is small and discusses the rationale for the constrained specification and references the core reference docs accordingly. For instance, no need to have all the step examples re-written.

I believe that this will make it easier for users to say the following:

“Okay, I get it. Open a RemoteConnection from a GraphTraversalSource (g.withRemote(...)) and spawn traversals and get back results. Thats it?! Easy peasy lemon squeezy.”

I believe that this will make it easier for system developers to say the following:

“We can cover 95% of use cases by simply exposing a RemoteConnection. Thats it?! Easy peasy lemon squeezy."

In general, this model looks at TinkerPop as a “remote database system” where users submit “queries” and get back “results.” That is it. Nothing Java-specific, nothing with executing arbitrary scripts remotely, nothing with REST, nothing with embedded (at least from a look-and-feel perspective), etc.

Thoughts?,
Marko.

http://markorodriguez.com



pieter-gmail

unread,
Nov 30, 2016, 11:19:05 AM11/30/16
to gremli...@googlegroups.com
Hi,

I generally recommend the exact opposite. Neo4j shines when embedded.
Sqlg already works of databases that is designed for remote access.
Wrapping that with yet another server is an unnecessary complication.
(Not having used it I do not know the usability or performance impact,
however being another layer it can only degrade performance and
complexity).

Further because Sqlg already works of a remote database it supports many
jvm's directly accessing the same graph. I imagine other providers could
have the same feature.

So gremlin-x is fine but it is not the obvious best choice for all
architectures.

All the above is for happy java devs.

Cheers
Pieter

Marko Rodriguez

unread,
Nov 30, 2016, 12:31:22 PM11/30/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
Hi,

I generally recommend the exact opposite. Neo4j shines when embedded.

And thats fine. There would simply be a EmbeddedRemoteConnection which is basically just a proxy that does nothing. No need for server stuff.

Sqlg already works of databases that is designed for remote access.

And yes, thats where a full fledged RemoteConnection (like GremlinServer’s) is needed for marshaling data over a network.

Wrapping that with yet another server is an unnecessary complication.

Its not about GremlinServer so much as its about RemoteConnection. If Sqlg has their own RemoteConnection implementation, then it uses that. Likewise, Neo4jServer could provide its own RemoteConnection implementation and thus, no need for GremlinServer. 

(Not having used it I do not know the usability or performance impact,
however being another layer it can only degrade performance and
complexity).

For embedded, its an implementation that just returns the traversal as was submitted.
For remote, its as it is now. You have to send the bytecode over the network and stream back results.

Further because Sqlg already works of a remote database it supports many
jvm's directly accessing the same graph. I imagine other providers could
have the same feature.

This is fine with RemoteConnection. What is “not fine” (the reason for gremlin-x) is to make it so users can’t talk with the Graph object. That is a Java thing and should not be exposed to users. More specifically, that is a Gremlin traversal machine <-> graph system provider interface. Users only interact with a RemoteConnection implementation.

So gremlin-x is fine but it is not the obvious best choice for all
architectures.

I was shootin’ this idea around DSEGraph people and Bob was like: “pfff…where was this idea 4 years ago. Everyone is already addicting to sending a Gremlin-Groovy script over the network and getting a ResultSet back.”

All the above is for happy java devs.

Again, the purpose is the constrain Java developers too. Things like this should never happen:

v1.property(‘name’).value()
v1.edges(OUT).next()
cluster.submit(“1+2”)
graph = GraphFactory.open(...)
graph.io(gryo()).readGraph(‘data/blah.gryo’)


It should be:

g.V(1).values(‘name’)
g.V(1).outE().next()
g.inject(1,2).sum()
g.withRemote(config)
// hmmm… don’t know about the last one. 

See ya,
Marko.


--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

pieter-gmail

unread,
Nov 30, 2016, 1:11:50 PM11/30/16
to gremli...@googlegroups.com
Ok, if Sqlg just implements its own RemoteConnection and no need for an
additional server in the architecture then I am a lot less nervous about
gremlin-x.

As an aside, Sqlg has its own peculiarities and optimization outside the
standard TinkerPop interface. Most providers will have such features. If
the graph is not exposed directly there will need to be some way to call
custom features. No idea how now, but still.

E.g. for Sqlg the most used such feature is
SqlgGraph.tx().batchMode(batchModeType) but there are others. Index
management will be another. Without being able to access these features
too many benefits of choosing a particular provider will be lost.

Cheers
Pieter
> graph.io <http://graph.io>(gryo()).readGraph(‘data/blah.gryo’)
>> <mailto:gremlin-user...@googlegroups.com>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-user...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer>.

Stephen Mallette

unread,
Nov 30, 2016, 4:58:09 PM11/30/16
to Gremlin-users, d...@tinkerpop.apache.org
Added back "dev"....

I think we've tried a lot of things for TinkerPop 3 and that there are too many options for application development as well as an over-exposed API, so I like this direction, though a fair bit of complexity lies in making it happen. Before getting to that complexity, I don't really think of this as a "new idea". To me the description of gremlin-x is really just a GLV for Java. I mean - consider that gremlin-python (something we classify as a GLV) is basically "everything a python dev needs to build applications - nothing more/less" and that's pretty much what we want for java in gremlin-x. gremlin-x would basically be "everything a java dev needs to build applications - nothing more/less". So I'm not seeing much distinction there (nor the need for naming it "gremlin-x" really - seems to be as simple as "gremlin-java" to be in line with other GLVs we will have).

Back to the "complexity" - and I think the questions in my mind that follow apply with or without this gremlin-x idea (they are general issues that just are faced by "remote" and GLVs):

+ I'll come back to this in the points below, but as a big fat general question: how will we deal with transactions?
+ What will be the nature of Graph.io() - in one sense i'd like to see that as something hidden as it is the source of a fair bit of confusion (i.e. someone trying to load a multi GB graphson file) but on the other hand it's convenient for small graphs - that story could be made better. Maybe Graph.io() being a part of "Graph" is hidden and only used by the test suite to load data into graph providers implementations for testing purposes. Of course, we will need our story in order for loading data in different formats if that isn't available (which I guess we need to do anyway).
+ Can remoting Gremlin bytecode cover everything that users currently do in embedded/local mode? The drawback with a remote traversal is that it is bound to a single transaction and you therefore lose some efficiency with certain graphs databases as logic with three traversals that might span a single transaction, for example, will be executed less efficiently as three different transaction refreshing a transaction cache each time. Maybe the server protocol needs to be session based? Maybe, going back to the first bullet i had, the transaction model needs to change so that its not bound to a single thread by default?  
+ Related to the last point, is there an easy way to blend client code with server code? Meaning, if i have a body of complex logic right now, my only option is to submit a script. I think the ScriptEngine stuff should really just be used for those situations where you need a lambda. That's a nice, simple, easy-to-explain rule, whereas "do it if there is complex logic" is a bit more subjective (and potentially troublesome). How can GLVs get complex logic on the "server" (let's say the logic is written in java and available to gremlin server) so that it could be executed as part of some bytecode in any GLV?
+ It would be neat to see what kind of "server" changes we could make to optimize for bytecode. Right now we sorta tied bytecode execution into the same Gremlin Server execution pipeline as ScriptEngine processing. I think we could probably do better on that.

Not sure there are any immediate answers to any of the above, but it's what I've been thinking about with respect to TinkerPop 3.3.0 lately.



>> <mailto:gremlin-users+unsub...@googlegroups.com>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-users+unsubscribe@googlegroups.com
> <mailto:gremlin-users+unsub...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/1b1639d8-4c1d-ed65-5218-8c8786b76c97%40gmail.com.

Marko Rodriguez

unread,
Dec 1, 2016, 9:44:06 AM12/1/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
Hi,

I think we've tried a lot of things for TinkerPop 3 and that there are too many options for application development as well as an over-exposed API, so I like this direction, though a fair bit of complexity lies in making it happen. Before getting to that complexity, I don't really think of this as a "new idea". To me the description of gremlin-x is really just a GLV for Java. I mean - consider that gremlin-python (something we classify as a GLV) is basically "everything a python dev needs to build applications - nothing more/less" and that's pretty much what we want for java in gremlin-x. gremlin-x would basically be "everything a java dev needs to build applications - nothing more/less". So I'm not seeing much distinction there (nor the need for naming it "gremlin-x" really - seems to be as simple as "gremlin-java" to be in line with other GLVs we will have).

Ahhhhhhhhhhhhhhhh… Okay, then there is no such thing as gremlin-core/ or gremlin-server/. Check out my packaging and subsequent reasoning:

gremlin-java/
  structure/
    io/
  process/
    remote/
      client/
      server/
    language/
      graph/
    machine/
      traversal/
        steps/
        strategies/
        computer/
        distributed/ **forthcoming**

And now, check out gremlin-python (as it currently stands):

gremlin-python/
  structure/
    io/
  process/
    remote/
      client/
    language/
      graph/

Take home points:

1. Gremlin-Java has a language, machine, server, and client implementation.
2. Gremlin-Python only has a language implementation and a remote client implementation.
3. remote/client is RemoteConnection and remote/server is RemoteServer (making up that interface for now).
4. We make more explicit the distinction between traversal machine and traversal language and now you know which variants have machine implementations.
5. In the future, Gremlin-Python could have a machine implementation and then would be native for Python-based graph systems.
- Thus, easy to grow TinkerPop into other languages.

In this way, if you are a graph system provider with a system written in language XXX, then you come to TinkerPop and ask:

1. Does gremlin-XXX exist?
2. If so, does gremlin-XXX have a machine/ implementation?
3. If so, then implement gremlin-XXX structure API and you are now TinkerPop-enabled.

In this way, if you are a graph system users with an application written in language XXX, then you come to TinkerPop and ask:

1. Does gremlin-XXX exist?
2. If so, does gremlin-XXX have a language/ implementation?
3. If so, then code in gremlin-XXX over any TinkerPop-enabled graph system.

Finally, this makes making gremlin-test/ more language agnostic all the more important. If all the gremlin-XXX languages can be verified “natively” using gremlin-test/ (not via ScriptEngine emulation), then we are super cool.

Finally, cause the last finally wasn’t enough, I would make it so gremlin-driver is in gremlin-java/ and I would keep gremlin-console/ as its own thing because, while it is a Java-based REPL, it is language agnostic given the current work by Stephen to enable any ScriptEngine (not just Gremlin-Groovy) to be leveraged in Gremlin Console.

Boom.

Marko.


To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H4390oFKxksw1bxDSO7Ej83NBes6NaBD_EBG77wD6QC786Q%40mail.gmail.com.

pieter-gmail

unread,
Dec 1, 2016, 10:07:00 AM12/1/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
Hi,

You say "v1.property(‘name’).value()" should never happen.
This is something we use all the time. Almost all the work we do is with
vertices and edges and is natural to call
element.property("something").isPresent(),
element.property("something").value(), element.property("something",
"somethingnew")

With this current train of thought will the use
Graph/Vertex/Edge/Property be discouraged or disappear?

Even as things stands now are you saying
vertex.property("something").value() should be replaced with
g.V(vertex).values("something")?

Thanks
Pieter


On 01/12/2016 16:43, Marko Rodriguez wrote:
> Hi,
>
>> + What will be the nature of Graph.io <http://graph.io>() - in one
>> sense i'd like to see that as something hidden as it is the source of
>> a fair bit of confusion (i.e. someone trying to load a multi GB
>> graphson file) but on the other hand it's convenient for small graphs
>> - that story could be made better. Maybe Graph.io <http://graph.io>()
>> > graph.io <http://graph.io/> <http://graph.io
>> <http://graph.io/>>(gryo()).readGraph(‘data/blah.gryo’)
>> >>> http://markorodriguez.com <http://markorodriguez.com/>
>> >>>
>> >>>
>> >>>
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> >> Groups "Gremlin-users" group.
>> >> To unsubscribe from this group and stop receiving emails from it,
>> >> send an email to gremlin-user...@googlegroups.com
>> <mailto:gremlin-users%2Bunsu...@googlegroups.com>
>> >> <mailto:gremlin-user...@googlegroups.com
>> <mailto:gremlin-users%2Bunsu...@googlegroups.com>>.
>> >> To view this discussion on the web visit
>> >>
>> https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com
>> <https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com>.
>> >> For more options, visit https://groups.google.com/d/optout
>> <https://groups.google.com/d/optout>.
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups "Gremlin-users" group.
>> > To unsubscribe from this group and stop receiving emails from
>> it, send
>> > an email to gremlin-user...@googlegroups.com
>> <mailto:gremlin-users%2Bunsu...@googlegroups.com>
>> > <mailto:gremlin-user...@googlegroups.com
>> <mailto:gremlin-users%2Bunsu...@googlegroups.com>>.
>> > To view this discussion on the web visit
>> >
>> https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com
>> <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com>
>> >
>> <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer
>> <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer>>.
>> > For more options, visit https://groups.google.com/d/optout
>> <https://groups.google.com/d/optout>.
>>
>> --
>> You received this message because you are subscribed to the
>> Google Groups "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to gremlin-user...@googlegroups.com
>> <mailto:gremlin-users%2Bunsu...@googlegroups.com>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/1b1639d8-4c1d-ed65-5218-8c8786b76c97%40gmail.com
>> <https://groups.google.com/d/msgid/gremlin-users/1b1639d8-4c1d-ed65-5218-8c8786b76c97%40gmail.com>.
>> For more options, visit https://groups.google.com/d/optout
>> <https://groups.google.com/d/optout>.
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to gremlin-user...@googlegroups.com
>> <mailto:gremlin-user...@googlegroups.com>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/CAA-H4390oFKxksw1bxDSO7Ej83NBes6NaBD_EBG77wD6QC786Q%40mail.gmail.com
>> <https://groups.google.com/d/msgid/gremlin-users/CAA-H4390oFKxksw1bxDSO7Ej83NBes6NaBD_EBG77wD6QC786Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-user...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/FF606E7A-446F-4A4D-A3A0-BC8B2EEAEAA7%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/FF606E7A-446F-4A4D-A3A0-BC8B2EEAEAA7%40gmail.com?utm_medium=email&utm_source=footer>.

Jason Plurad

unread,
Dec 1, 2016, 10:20:49 AM12/1/16
to Gremlin-users, d...@tinkerpop.apache.org
> + I'll come back to this in the points below, but as a big fat general question: how will we deal with transactions?

Bullseye. I'd add that assuming that TinkerGraph would be the "one true TinkerPop graph" for gremlin-x testing, it needs to support transactions which it doesn't do currently. Or introduce a new reference implementation graph that does.


>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-users+unsubscribe@googlegroups.com

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/1b1639d8-4c1d-ed65-5218-8c8786b76c97%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H4390oFKxksw1bxDSO7Ej83NBes6NaBD_EBG77wD6QC786Q%40mail.gmail.com.

Marko Rodriguez

unread,
Dec 1, 2016, 10:53:47 AM12/1/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
Hello,

You say "v1.property(‘name’).value()" should never happen.
This is something we use all the time. Almost all the work we do is with
vertices and edges and is natural to call
element.property("something").isPresent(),
element.property("something").value(), element.property("something",
"somethingnew”)


So “we” as “provider” or “we” as “user”?
* If provider, of course, internally, your TinkerPop implementation and supporting code can use direct structure API calls.
* If user, then no, users should never talk to the structure API, only the process API (i.e., traversal).

Why so strict for “users"? A few reasons:

1. Think about Gremlin-Python. Users only gets back “proxy” vertices/edges/etc. Thus, those users won’t be able to do what Java users can. Inconsistent experience/documentation.
2. Vertex/Edge/etc. methods are only useful in embedded systems as remotely, even Gremlin-Java users just get proxies. Again, inconsistent experience/documentation.
3. We should not frame interactions with a graph system in terms of an “API” as much as in terms of a “database connection.”
- So while MySQL JDBC returns row and column objects in a ResultSet, they are just proxy containers of the data in MySQL. Mutating a ResultSet doesn’t alter the MySQL data.

With this current train of thought will the use
Graph/Vertex/Edge/Property be discouraged or disappear?

For users, yes, save that they will get “proxy objects” like ReferenceVertex/ReferenceEdge/etc. I strongly disagree with our use of DetachedVertex/DetachedEdge/etc. for the reason that it exposes both too much data (bandwidth) and DetachedXXX objects are intended to be interacted with via the structure API — e.g. vertex.property(“name”).isPresent().

Even as things stands now are you saying
vertex.property("something").value() should be replaced with
g.V(vertex).values("something”)?

For users, yes.

Note that there is one problem:

Graph.addVertex() is way faster than g.addV(). Why? Graph.addVertex() is a direct method call to Graph without the complication of traversal compilation and Traverser encapsulation. We need to make the graph traversal way of CRUD to a graph system (nearly) as fast as direct structure API calls. Solutions:

1. A MutationStrategy that ultimately compile g.addV() to just Graph.addVertex() without all the fluff of Step semantics.
2. Strategy loading should be tied to step usage. In other words, why would we call MatchPredicateStrategy on a traversal that is g.addV()? The use of match() should trigger the loading of that strategy.
3. Make strategy application faster as it currently stands. This is linear improvement, but still, why not.

Thoughts?,
Marko.



To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/7935b777-bcec-4973-7e56-1b476944f0c5%40gmail.com.

Robert Dale

unread,
Dec 1, 2016, 12:32:22 PM12/1/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
As someone who has gone through the trouble to migrate all scripts to Java withRemote, I'll throw in my $0.02.

I agree the focus should be on the Connection (being separate from Graph) and Traversal. I wouldn't constrain it to "RemoteConnection", just Connection or GraphConnection. Perhaps there's an EmbeddedConnection and a RemoteConnection or maybe it's URI-oriented similar to how JDBC does it. In either case, the behavior  of Remote and Embedded is the same which is what I think we're striving for.

I would also like to see Transactions be Connection-oriented. With the right API, it could hook into JTA and be able to take advantage of various annotations for marking transaction boundaries.

An example of Connection:

Connection con = new GraphConnection("gremlin:server://localhost:8182/database"); // returns RemoteConnection
// con = new GraphConnection("gremlin:tinkergraph:in-mem/database"); // returns EmbeddedConnection
// con = new GraphConnection("gremlin:neo4j:file://tmp/neo/database");

con.setAutoCommit(false)
con.tx().begin()
g = con.traversal()
g.addV().next()
con.tx().commit()

Connection could also submit scripts. I still use scripts to access Graph for db migration and performing schema updates.
con.submit("g.V()").all()

Something that should help performance and might help get away from general scripts is batching. Send all traversals at once:

conn.batch( 
  g.addV(),
  g.V().count(),
  g.V()
).iterate()

Are there features of a lambda that couldn't be replaced by a more feature-rich gremlin? 
g.V().out('knows').map{it.get().value('name') + ' is the friend name'}
g.V().out('knows').map(lambda(concat(__.it.get().value('name'), ' is the friend name'))

Reference-only makes total sense. This works really well especially with a local cache or for use cases where most of the data is stored in a separate database. I think it would lend itself nicely to lazy loading. When you need values there are options for that as well (properties/values/valueMap).  One of the problems with 'attached' elements is you don't know what the implementation does. So potentially every get or set property call is going to the database and you don't realize it. That can hurt performance and have unintended consequences.

For loading graphs, Connection could also load and send raw GraphSON
conn.load(new String(Files.readAllBytes(Paths.get("gremlin.gson")))) // perhaps a Stream would be better
But maybe that should be reserved for the graph side.

Still there should to be some sort of schema management interface or some way of accessing the Graph for live updates. Think when the database is embedded in a remote gremlin server (ala Neo4j). Otherwise the gremlin server would have to be shutdown, restart with native tools, perform work, shutdown, restart gremlin server. Maybe it's vendor-specific on the Connection implementation. Maybe it's just a limitation of using embedded DBs going forward (Neo4j 3.x has remote capabilities). Maybe this is moot if the GLV is the machine.  Just something to consider.


Robert Dale

Hello,


To view this discussion on the web visit

   https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com
   <https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com>.
For more options, visit https://groups.google.com/d/optout
   <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google
Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from
   it, send

To view this discussion on the web visit

   https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com
   <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com>

   <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer
   <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer>>.
For more options, visit https://groups.google.com/d/optout
   <https://groups.google.com/d/optout>.

   --
   You received this message because you are subscribed to the
   Google Groups "Gremlin-users" group.
   To unsubscribe from this group and stop receiving emails from it,

   To view this discussion on the web visit
   https://groups.google.com/d/msgid/gremlin-users/1b1639d8-4c1d-ed65-5218-8c8786b76c97%40gmail.com
   <https://groups.google.com/d/msgid/gremlin-users/1b1639d8-4c1d-ed65-5218-8c8786b76c97%40gmail.com>.
   For more options, visit https://groups.google.com/d/optout
   <https://groups.google.com/d/optout>.



-- 
You received this message because you are subscribed to the Google
Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it,

-- 
You received this message because you are subscribed to the Google
Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send

-- 
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/315C4503-D935-44F7-B7C2-98D5BC22D345%40gmail.com.

Stephen Mallette

unread,
Dec 1, 2016, 12:57:33 PM12/1/16
to Gremlin-users, d...@tinkerpop.apache.org

> Something that should help performance and might help get away from general scripts is batching

I've thought about batching as a feature many times, but i'm not sure it solves the use case folks typically have that causes us problems. If most batches looked like the one in your example that would be excellent, but I think most "complex logic" tend to contain variables and and other looping logic that you'd like to see as part of the batch. But maybe that's a separate problem - if i have three unrelated traversals submitting them in batch would be nice.

> Connection could also submit scripts. I still use scripts to access Graph for db migration and performing schema updates.

hmm - we often forget the need for schema management. a key problem with scripts is the security angle. we have a sandbox but it is bound to groovy and i often wonder how well it could pass a rigorous security audit. It also has the downside that it will force you to alter how you write scripts in that they will be type-checked on evaluation.

In an effort to keep remoting about bytecode, perhaps we leverage some of the work already done with lambdas. Somehow, you could initialize a remote traversal with a lambda that could have an arbitrary script? in that way it's still sent as bytecode and not another layer to the protocol? Perhaps that same functionality could provide a way to access "complex logic" (i.e. stored procedures) on the server in much the way that Neo4j has its "CALL" in Cypher:

CALL org.neo4j.examples.findDenseNodes(1000)

Maybe there could be some sort of remote "call" step that let's us do something like that. Sorry - this email is all over the place - just tossing things out there.





Robert Dale

Hello,

   <mailto:gremlin-users%2Bunsubsc...@googlegroups.com>>.
To view this discussion on the web visit

   https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com
   <https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com>.
For more options, visit https://groups.google.com/d/optout
   <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google
Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from
   it, send
an email to gremlin-users+unsubscribe@googlegroups.com
   <mailto:gremlin-users%2Bunsubsc...@googlegroups.com>
   <mailto:gremlin-users%2Bunsubsc...@googlegroups.com>>.
To view this discussion on the web visit

   https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com
   <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com>

   <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer
   <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer>>.
For more options, visit https://groups.google.com/d/optout
   <https://groups.google.com/d/optout>.

   --
   You received this message because you are subscribed to the
   Google Groups "Gremlin-users" group.
   To unsubscribe from this group and stop receiving emails from it,
   send an email to gremlin-users+unsubscribe@googlegroups.com
   <mailto:gremlin-users%2Bunsubsc...@googlegroups.com>.
   To view this discussion on the web visit
   https://groups.google.com/d/msgid/gremlin-users/1b1639d8-4c1d-ed65-5218-8c8786b76c97%40gmail.com
   <https://groups.google.com/d/msgid/gremlin-users/1b1639d8-4c1d-ed65-5218-8c8786b76c97%40gmail.com>.
   For more options, visit https://groups.google.com/d/optout
   <https://groups.google.com/d/optout>.



-- 
You received this message because you are subscribed to the Google
Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to gremlin-users+unsubscribe@googlegroups.com

-- 
You received this message because you are subscribed to the Google
Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to gremlin-users+unsubscribe@googlegroups.com

-- 
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/7935b777-bcec-4973-7e56-1b476944f0c5%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/315C4503-D935-44F7-B7C2-98D5BC22D345%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CABed_4pqU_SJjEvRjuy%2BdNOxtZoApyxVH05rDiTUsxWpQaqbcg%40mail.gmail.com.

pieter-gmail

unread,
Dec 1, 2016, 1:04:52 PM12/1/16
to gremli...@googlegroups.com
I was talking about we as users.

One of the first reasons I came to graphs, Neo4j and then TinkerPop way
back was precisely because of the direct access to Node/Vertex. The user
treats it like any other object, not a remote connection. It is the
embedded nature that makes life so easy. In a way it was like having a
simplistic Hibernate as the core api. 99% of queries we write is to
retrieve vertices. Not Maps and Lists of something. TinkerPop's own test
suite applies this type of thinking. Querying/modifying Elements and
asserting them. Vertex and Edge abound as first class citizens.

Graph, Vertex and Edge is the primary abstraction that users deal with.
Having the direct representation of this is very very nice.

It makes user code easy and readable. You know you are dealing with the
"Person/Address/Dog/This/That" entity/label as opposed to just a
decontextualized bunch of data, Maps and Lists. If Vertex/Edge/Property
were to disappear I'd say it would be the first call of duty to write a
baby hibernate to bring the property model back in again into userspace.

Regarding jdbc, this kinda makes the point. Sqlg and Hibernate and many
many other tools exists precisely so that users do not need to use JDBC
with endless hardcoded strings guiding the application. Making TinkerPop
more like JDBC is not an obvious plus point.

A ReferencedElement is also no good as the problem I experience is
latency not bandwidth.

I reckon the experience and usage of TinkerPop is rather different for
java and non java people and perhaps even java folks. Hopefully I am not
the only one who have made such heavy happy use of the TinkerPop
property meta model and would be sad to see it go.

Cheers
Pieter
>>>> <http://graph.io/> <http://graph.io <http://graph.io/>>() - in one
>>>> sense i'd like to see that as something hidden as it is the source of
>>>> a fair bit of confusion (i.e. someone trying to load a multi GB
>>>> graphson file) but on the other hand it's convenient for small graphs
>>>> - that story could be made better. Maybe Graph.io
>>>> <http://graph.io/> <http://graph.io <http://graph.io/>>()
>>>> <mailto:pieter...@gmail.com> <mailto:pieter...@gmail.com>>
>>>>> graph.io <http://graph.io/> <http://graph.io/> <http://graph.io
>>>>>>> <http://markorodriguez.com/> <http://markorodriguez.com/>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Gremlin-users" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to gremlin-user...@googlegroups.com
>>>>>> <mailto:gremlin-user...@googlegroups.com>
>>>> <mailto:gremlin-users%2Bunsu...@googlegroups.com
>>>> <mailto:2Bunsu...@googlegroups.com>>
>>>>>> <mailto:gremlin-user...@googlegroups.com
>>>> <mailto:gremlin-users%2Bunsu...@googlegroups.com
>>>> <mailto:2Bunsu...@googlegroups.com>>>.
>>>>>> To view this discussion on the web visit
>>>>>>
>>>> https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com
>>>> <https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com>.
>>>>>> For more options, visit https://groups.google.com/d/optout
>>>> <https://groups.google.com/d/optout>.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Gremlin-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from
>>>> it, send
>>>>> an email to gremlin-user...@googlegroups.com
>>>>> <mailto:gremlin-user...@googlegroups.com>
>>>> <mailto:gremlin-users%2Bunsu...@googlegroups.com
>>>> <mailto:2Bunsu...@googlegroups.com>>
>>>>> <mailto:gremlin-user...@googlegroups.com
>>>> <mailto:gremlin-users%2Bunsu...@googlegroups.com
>>>> <mailto:2Bunsu...@googlegroups.com>>>.
>>>>> To view this discussion on the web visit
>>>>>
>>>> https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com
>>>> <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com>
>>>>>
>>>> <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer
>>>> <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer>>.
>>>>> For more options, visit https://groups.google.com/d/optout
>>>> <https://groups.google.com/d/optout>.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the
>>>> Google Groups "Gremlin-users" group.
>>>> To unsubscribe from this group and stop receiving emails from it,
>>>> send an email to gremlin-user...@googlegroups.com
>>>> <mailto:gremlin-user...@googlegroups.com>
>>>> <mailto:gremlin-users%2Bunsu...@googlegroups.com
>>>> <mailto:2Bunsu...@googlegroups.com>>.
>> send an email to gremlin-user...@googlegroups.com
>> <mailto:gremlin-user...@googlegroups.com>.
>> To view this discussion on the web
>> visit https://groups.google.com/d/msgid/gremlin-users/7935b777-bcec-4973-7e56-1b476944f0c5%40gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-user...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/315C4503-D935-44F7-B7C2-98D5BC22D345%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/315C4503-D935-44F7-B7C2-98D5BC22D345%40gmail.com?utm_medium=email&utm_source=footer>.

Stephen Mallette

unread,
Dec 1, 2016, 1:33:49 PM12/1/16
to Gremlin-users, d...@tinkerpop.apache.org
adding back dev....

With Pieter's points in mind, Marko, ultimately does the "classic" embedded mode to using TinkerPop go away for JVM users? I mean - it's not as though those classes disappear. I suppose that in the end there's nothing we can really do to prevent someone on the JVM from operating at the level Pieter wants to operate at, right? 

To me, I would think that gremlin-x/gremlin-java (or however it all shakes out) is about exposing a body of code that provides the prescribed pattern for developing applications with TinkerPop. So our user documentation would cease to discuss what we've dubbed the "Structure API" and would instead focus exclusively on the "Process API" ("structure" stuff would move to the Provider Documentation). In many ways, I think that's a direction we've been heading anyway. All the latest recipes have been written that way. I've noticed SO and mailing list answers being written that way. It hasn't been 100% consistent in terms of messaging but I guess we would look to use this pivot point to start to do that.



>>>> <mailto:pieter.martin@gmail.com> <mailto:pieter.martin@gmail.com>>
>>>>>> send an email to gremlin-users+unsubscribe@googlegroups.com
>>>>>> <mailto:gremlin-users+unsub...@googlegroups.com>
>>>>    <mailto:gremlin-users%2Bunsu...@googlegroups.com
>>>> <mailto:2Bunsubscribe@googlegroups.com>>
>>>>>> <mailto:gremlin-users+unsub...@googlegroups.com
>>>>    <mailto:gremlin-users%2Bunsu...@googlegroups.com
>>>> <mailto:2Bunsubscribe@googlegroups.com>>>.
>>>>>> To view this discussion on the web visit
>>>>>>
>>>>    https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com
>>>>    <https://groups.google.com/d/msgid/gremlin-users/6f8790d7-9fa8-99c8-b553-1793c42f9c04%40gmail.com>.
>>>>>> For more options, visit https://groups.google.com/d/optout
>>>>    <https://groups.google.com/d/optout>.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Gremlin-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from
>>>>    it, send
>>>>> an email to gremlin-users+unsubscribe@googlegroups.com
>>>>> <mailto:gremlin-users+unsub...@googlegroups.com>
>>>>    <mailto:gremlin-users%2Bunsu...@googlegroups.com
>>>> <mailto:2Bunsubscribe@googlegroups.com>>
>>>>> <mailto:gremlin-users+unsub...@googlegroups.com
>>>>    <mailto:gremlin-users%2Bunsu...@googlegroups.com
>>>> <mailto:2Bunsubscribe@googlegroups.com>>>.
>>>>> To view this discussion on the web visit
>>>>>
>>>>    https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com
>>>>    <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com>
>>>>>
>>>>    <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer
>>>>    <https://groups.google.com/d/msgid/gremlin-users/50D34645-5AF8-4683-8C1F-E01A3C4F7815%40gmail.com?utm_medium=email&utm_source=footer>>.
>>>>> For more options, visit https://groups.google.com/d/optout
>>>>    <https://groups.google.com/d/optout>.
>>>>
>>>>    --
>>>>    You received this message because you are subscribed to the
>>>>    Google Groups "Gremlin-users" group.
>>>>    To unsubscribe from this group and stop receiving emails from it,
>>>>    send an email to gremlin-users+unsubscribe@googlegroups.com
>>>> <mailto:gremlin-users+unsub...@googlegroups.com>
>>>>    <mailto:gremlin-users%2Bunsu...@googlegroups.com
>>>> <mailto:2Bunsubscribe@googlegroups.com>>.
>>>>    To view this discussion on the web visit
>>>>    https://groups.google.com/d/msgid/gremlin-users/1b1639d8-4c1d-ed65-5218-8c8786b76c97%40gmail.com
>>>>    <https://groups.google.com/d/msgid/gremlin-users/1b1639d8-4c1d-ed65-5218-8c8786b76c97%40gmail.com>.
>>>>    For more options, visit https://groups.google.com/d/optout
>>>>    <https://groups.google.com/d/optout>.
>>>>
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Gremlin-users" group.
>>>> To unsubscribe from this group and stop receiving emails from it,

>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/gremlin-users/CAA-H4390oFKxksw1bxDSO7Ej83NBes6NaBD_EBG77wD6QC786Q%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/gremlin-users/CAA-H4390oFKxksw1bxDSO7Ej83NBes6NaBD_EBG77wD6QC786Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Gremlin-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send

>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/gremlin-users/FF606E7A-446F-4A4D-A3A0-BC8B2EEAEAA7%40gmail.com
>>> <https://groups.google.com/d/msgid/gremlin-users/FF606E7A-446F-4A4D-A3A0-BC8B2EEAEAA7%40gmail.com?utm_medium=email&utm_source=footer>.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to gremlin-users+unsubscribe@googlegroups.com
>> <mailto:gremlin-users+unsub...@googlegroups.com>.

>> To view this discussion on the web
>> visit https://groups.google.com/d/msgid/gremlin-users/7935b777-bcec-4973-7e56-1b476944f0c5%40gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-users+unsubscribe@googlegroups.com
> <mailto:gremlin-users+unsub...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/315C4503-D935-44F7-B7C2-98D5BC22D345%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/315C4503-D935-44F7-B7C2-98D5BC22D345%40gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/bd86a1f0-a8ff-e9b6-f15e-62791fec2622%40gmail.com.

Marko Rodriguez

unread,
Dec 1, 2016, 1:38:36 PM12/1/16
to gremli...@googlegroups.com
Hi,

PIETER REPLIES:

One of the first reasons I came to graphs, Neo4j and then TinkerPop way
back was precisely because of the direct access to Node/Vertex. The user
treats it like any other object, not a remote connection. It is the
embedded nature that makes life so easy. In a way it was like having a
simplistic Hibernate as the core api. 99% of queries we write is to
retrieve vertices. Not Maps and Lists of something. TinkerPop's own test
suite applies this type of thinking. Querying/modifying Elements and
asserting them. Vertex and Edge abound as first class citizens.

So Graph/Vertex/Edge/VertexProperty/Property will still exist for users as objects in the respective GLV language, it is just they are not “attached” and “rich.”

For instance, in Gremlin-Python, you have:

v = g.V().next()
A ReferenceVertex contains the id of the vertex so you can always “re-attach” it to the source.

g.V(v).out()

Graph, Vertex and Edge is the primary abstraction that users deal with.
Having the direct representation of this is very very nice.
It makes user code easy and readable.  You know you are dealing with the
"Person/Address/Dog/This/That" entity/label as opposed to just a
decontextualized bunch of data, Maps and Lists. If Vertex/Edge/Property
were to disappear I'd say it would be the first call of duty to write a
baby hibernate to bring the property model back in again into userspace.

Again, the abstraction is still there, but just ALWAYS in a detached form.


Regarding jdbc, this kinda makes the point. Sqlg and Hibernate and many
many other tools exists precisely so that users do not need to use JDBC
with endless hardcoded strings guiding the application. Making TinkerPop
more like JDBC is not an obvious plus point.

So, RemoteConnection differs from JDBC in that its not a fat string, but RemoteConnection.submit(Bytecode). Thus, you still work at the GraphTraversal level in every GLV.

A ReferencedElement is also no good as the problem I experience is
latency not bandwidth.

So with ReferenceElements, latency will be less too because it takes less time to construct the ReferenceVertex than it does to construct a DetachedVertex. Imagine a vertex with 100 properties and meta properties. ?!

I reckon the experience and usage of TinkerPop is rather different for
java and non java people and perhaps even java folks. Hopefully I am not
the only one who have made such heavy happy use of the TinkerPop
property meta model and would be sad to see it go.

Cheers
Pieter



ROBERT REPLIES:

I agree the focus should be on the Connection (being separate from Graph) and Traversal. I wouldn't constrain it to "RemoteConnection", just Connection or GraphConnection. Perhaps there's an EmbeddedConnection and a RemoteConnection or maybe it's URI-oriented similar to how JDBC does it. In either case, the behavior  of Remote and Embedded is the same which is what I think we're striving for.

Yes. Good point. Just Connection.

I would also like to see Transactions be Connection-oriented. With the right API, it could hook into JTA and be able to take advantage of various annotations for marking transaction boundaries.

g = g.openTx()
g.V().out().out()
g.addV()
g.V(1).addE().to(2)
g.closeTx();

??? This way, its all about GraphTraversalSource/GraphTraversal. That is truly the “connection” where the Connection implementation is just provider/machine specific shuffling of Bytecode in and Traversers out.

Are there features of a lambda that couldn't be replaced by a more feature-rich gremlin? 
g.V().out('knows').map{it.get().value('name') + ' is the friend name'}
g.V().out('knows').map(lambda(concat(__.it.get().value('name'), ' is the friend name’))

So we currently have the concept of g:Lambda and this allows for lambdas to be used remotely.

g.V().map(function(“it.get().label()”)) // Gremlin-Java traversal with a Gremlin-Groovy lambda.

The crappy thing is that the lambda is always a String.

Reference-only makes total sense. This works really well especially with a local cache or for use cases where most of the data is stored in a separate database. I think it would lend itself nicely to lazy loading. When you need values there are options for that as well (properties/values/valueMap).  One of the problems with 'attached' elements is you don't know what the implementation does. So potentially every get or set property call is going to the database and you don't realize it. That can hurt performance and have unintended consequences.

Dude, I’ve been saying this forever. DetachedXXX is a bad idea for the reasons you have stipulated. Just imagine:

g.V(1).out(‘knows')

The GraphSON return is every vertex 1 knows and all its properties and meta properties?!?! If you wanted that data too you would have queried for it.

Marko.

pieter-gmail

unread,
Dec 1, 2016, 2:29:05 PM12/1/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
Hi,

"So with ReferenceElements, latency will be less too because it takes
less time to construct the ReferenceVertex than it does to construct a
DetachedVertex. Imagine a vertex with 100 properties and meta
properties. ?!"

But ReferencedElement does not have the properties so more round trips
are needed increasing latency. One of the first things to make Sqlg at
all usable was to make sure that a Vertex contains all of its
properties. Else at least one more call is needed per Vertex. Its a
latency killer. For those mostly few cases where the Vertex is so fat
that it is slow to load and only a few properties are needed then
g.V().hasLabel("label").values("property1", "property2") is used. So to
my mind ReferencedElement increases latency and does not decreases it.

Using ReferencedElement which is hardly an Element at all, after all it
throws exceptions on almost all of its own interface, the user has to
get the properties manually and then is back in a world of Map and Lists
of Maps.

A refactor of much existing code will need to toss the Vertex notion all
together and replace it with Maps and Lists of Maps. Almost like writing
an application in pure JDBC code with thousands of lines iterating
through ResultSets mapping things back and forth. Unless I am missing
something this change seems huge.

I get that all this is important for non java devs but it be a pity if
their problems becomes java devs problems.

Cheers
Pieter


On 01/12/2016 20:38, Marko Rodriguez wrote:
> Hi,
>
> *PIETER REPLIES:*
> *ROBERT REPLIES:*
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-user...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer>.

Marko Rodriguez

unread,
Dec 1, 2016, 3:57:22 PM12/1/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
Ah. Yes. I concur Pieter.

Then I think we need to get smart about “what data?” We sort of already do with HaltedTraverserStrategy but this is really specific to internal computing and GremlinServer. We could go deeper into this path with:

g.withDetachment(…)
  - Detach.reference // just ids
  - Detach.reduced // ids, labels, keys, values (basically, what you need for toString())
  - Detach.rich // property data included
  - Detached.full // edge data included (basically, StarGraph)

Next, GraphSON would have to specify a “subtype” to the various g:Element types.

{ @type:”g:Vertex", @detach:”reference", @value: { id:1 } }

Then, we could add a method to Element.

Element.detachment() -> Detach enum

This way, users can always know what is available to them. Finally for cleanliness:

Element.detach(Detach) -> Element
Element.attach(Graph) -> Element (always a Detach.full element)

Just spit-ballin’,
Marko.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com.

Stephen Mallette

unread,
Dec 1, 2016, 5:42:33 PM12/1/16
to Gremlin-users, d...@tinkerpop.apache.org
I don't think HaltedTraverserStrategy is a good way to deal with properties vs no properties. It really should be a function of the serializer. I say that because I seem to remember that if the Vertex (or Edge or whatever) is embedded in another object then the HaltedTraverserStrategy won't properly do your bidding. Like:

g.V().groupCount()

will embed the Vertex in a Map. ReferenceFactory and DetachedFactory don't traverse the tree of objects when detaching right now - a serializer however will do that in the normal course of its work. So, perhaps withDetachment() would hint to the serializer what kind of detachment it wants.


--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/F51985DD-7F40-45D7-BA18-FEC81E130099%40gmail.com.

pieter-gmail

unread,
Dec 2, 2016, 6:49:41 AM12/2/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
Hi,

Perhaps this is also the time to think about custom steps and some way
for providers to inject custom steps into the traversal.
Currently the steps are all defined on `GraphTraversal`. Maybe providers
can extend the interface with their functionality and if
`graph.traversal()` returns the extended interface the custom steps will
be available.

e.g.

graph.traversal().streamVertex(...)

Or if the traversal becomes session aware then perhaps,

GraphTraversal gt = graph.traversal();
gt.bulkModeOn();
for 1 .. 1 000 000
gt.addV().next();

gt.V().specialRecipe().next();
gt.commit();

Just some thoughts

Cheers
Pieter
>> send an email to gremlin-user...@googlegroups.com
>> <mailto:gremlin-user...@googlegroups.com>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-user...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/F51985DD-7F40-45D7-BA18-FEC81E130099%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/F51985DD-7F40-45D7-BA18-FEC81E130099%40gmail.com?utm_medium=email&utm_source=footer>.

Stephen Mallette

unread,
Dec 2, 2016, 7:04:01 AM12/2/16
to Gremlin-users, d...@tinkerpop.apache.org
> Perhaps this is also the time to think about custom steps and some way for providers to inject custom steps into the traversal.

I'm not completelyclear on how custom steps, DSLs, etc should be properly implemented. I think it would be important to have an approach for that under this model and get it documented.


>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com
>>> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer>.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Gremlin-users" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to gremlin-users+unsubscribe@googlegroups.com
>> <mailto:gremlin-users+unsub...@googlegroups.com>.

>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-users+unsubscribe@googlegroups.com
> <mailto:gremlin-users+unsub...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/F51985DD-7F40-45D7-BA18-FEC81E130099%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/F51985DD-7F40-45D7-BA18-FEC81E130099%40gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/7dc4ff30-0730-46ef-befc-ae8d2bf36e8d%40gmail.com.

Robert Dale

unread,
Dec 2, 2016, 7:57:43 AM12/2/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
Pieter, while I think Marko may be onto something, I just want to completely disagree with you as a Java dev. ;-)

First, in Neo4j's impl, from what I can tell the elements are not fully loaded. Every get (getProperty, edges, etc) does a query to the database. This is more round trips to the database. So this is why I made the statement that implementations are different.  In your sqlg case, you are basically arguing that the default behavior is the sql equivalent of SELECT *.  This is not a good practice. Then you go on to say that if the dev is aware that this is a 'fat' element, they should ask for exact properties.  I think what we're arguing is that the default behavior should be 'always ask for exact properties'. This is the most accepted practice in querying any database, sql, nosql, mongodb, cassandra, etc.

That leads us to your Hibernate comment.  In the abstract sense, Vertex is just a map wrapper. I think you're just splitting hairs trying to distinguish a Dog Vertex and a Dog Map. In either case, you would have to query the label.  In any case, I don't know anyone who wants to deal with Vertex/Edges.  What most devs deal with, in my experience, is a domain-specific model.  So whether I get back a Vertex or a Map, either way, I'm going to translate that to my domain model.  Also, in hibernate, when I get a property I didn't query for, I will get a null.  If I set a property, it does not automatically persist it to the database. In your model, there is no difference between transient, in-memory state (e.g. workflow) and database state. BTW, this would also be lots of round trips to the database in your case. Finally, believe it or not, Hibernate attempts to do smart querying where it will actually retrieve only the IDs, then look for them in its second-level cache, if not found, go back to the database to get them.  This is a very common pattern across sql/nosql datastores.

So it's not just about becoming more like jdbc but more about a low-level paradigm. To that I agree with you on one thing, the first thing you should do is create a 'baby hibernate' because I don't think gremlin should be an ORM (OGM?).



Robert Dale

> <mailto:gremlin-users+unsub...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com.

pieter-gmail

unread,
Dec 2, 2016, 10:30:34 AM12/2/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
Hi,

Let me disagree with your disagreement ;-)

Regarding Neo4j

I am talking about Neo4j embedded. The node/vertex is pretty much the
database already being a direct pointer to the node on disc with its
properties right next to it on disc. I would be surprised if all the
properties are not also already in its hot cache. I am speculating about
the internals but when coding in Neo4j embedded you don't care about
pre-loading all or some properties for performance reasons, just load
the node and all is well. Its the beauty of embedded Neo4j, latency is
just not a concern and the node represents a instance of a label.

It would be interesting to execute TinkerPop Neo4j's structure and
process test suites via gremlin server and compare the performance to
embedded. I don't really have a clue what to expect. If every property
access is to be a call via GremlinServer I reckon things will slow down
significantly. The suite is composed with the implicit assumption that
property access is not something to think about.

Regarding Hibernate. I have not worked with Hibernate for some time so
ran a test to make sure.

EntityManager entityManager =
entityManagerFactory.createEntityManager();
entityManager.getTransaction().begin();
int count = 100;
for (int i = 1; i < count + 1; i++) {
Person person = new Person("person_" + i);
entityManager.persist(person);
}
entityManager.getTransaction().commit();
entityManager.close();

entityManager = entityManagerFactory.createEntityManager();
Person person = entityManager.find(Person.class, 1L);
assertNotNull(person);
assertEquals("person_1", person.getName());

The entityManager.find(Person.class, 1L) resulted in the following sql.

"select person0_.id as id1_5_0_, person0_.name as name2_5_0_ from Person
person0_ where person0_.id=?"

I did not ask for the name property, it returned it anyways as well it
should. If every property needs to be gotten separately then latency
will kill the app.
If the user has to ask for every property individually, well then part
of the point of Hibernate disappears.

RE: "Vertex is just a map wrapper"
But its not just any map, its a Vertex, a core notion of the property
graph model.

RE: "I don't know anyone who wants to deal with Vertex/Edges"
We probably live in our own bubbles but I don't know anyone who would
not want to deal with the core abstractions of the property graph model
and rather deal with Maps, except perhaps Json/Javascript folks :-)

The property graph model and graph traversals are all about vertices and
edge traversals, having that right there as a first class citizen in
code is great.

RE: in hibernate "If I set a property, it does not automatically persist
it to the database."
True but its also the cause of pain with hibernate altogether bypassing
the databases concurrency model with it optimistic locking. And voilla
you are stuck with lets just ignore the exception and retry and hope we
get lucky this time round logic. For what its worth setting a property
on Sqlg runs a update statement. Alas a very good reason why Hibernate
does what it does is because their way reduces latency being able to run
batch updates on commit or flush. Sqlg supports batch updates but its
not the default.

RE: "In your model, there is no difference between transient, in-memory
state (e.g. workflow) and database state."
Not sure what you mean here. If you mean application writers keeping
their own cache of persistent data then you are right. Rule #1 of
caching is don't cache. Rule #2 is don't cache the cache. Caching is a
solution to a weakness elsewhere. I am not saying don't ever cache but
that if you can avoid it do so. Writing transactional caches is also a
rather specialized and difficult exercise and precisely what databases
are all about.

Lastly, to make sure we are talking about the same change, are you
proposing that all gremlins like

GraphTraversal<Vertex, Vertex> vertices =
this.sqlgGraph.traversal().V().out();

should become

GraphTraversal<Vertex, Map<String, Object>> vertexProperties =
this.sqlgGraph.traversal().V().out().valueMap();

or worse

GraphTraversal<Vertex, Map<String, Object>> vertexProperties =
this.sqlgGraph.traversal().V().out().values("propery1", "propety2",
"property3");

Cheers
Pieter
> > v.id <http://v.id>
> > an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>
> > <mailto:gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com>
> >
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer>>.
> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-user...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/CABed_4qE89f4oqZPQGjRXP8hn4kQpqVUiE%3DGq%2Bnvu_XfTQ_mWw%40mail.gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/CABed_4qE89f4oqZPQGjRXP8hn4kQpqVUiE%3DGq%2Bnvu_XfTQ_mWw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

pieter-gmail

unread,
Dec 3, 2016, 3:39:46 PM12/3/16
to gremli...@googlegroups.com
Hi,

Had some more thoughts regarding this.

If TinkerPop recipes were become steps somehow then providers can
optimize them individually. Currently some recipes, (complex gremlins in
general) are unoptimizeable. However if the recipe were to be a step
then the individual recipe/step could be optimized using a different
strategy to that of the default recipe's gremlin.

Providers are then also able to add their own recipes and if its popular
enough it could make it into TinkerPop's library of recipes via standard
gremlin.

Cheers
Pieter
> <pieter...@gmail.com <mailto:pieter...@gmail.com>
> >> <mailto:pieter...@gmail.com
> >>> v.id <http://v.id>
> >>> an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>
> >>> <mailto:gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>>
> >>> <mailto:gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>>.
> >>> To view this discussion on the web visit
> >>>
> https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com>
> >>>
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer>>.
> >>> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> >> Groups "Gremlin-users" group.
> >> To unsubscribe from this group and stop receiving emails from it,
> >> send an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>
> >> <mailto:gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>>.
> >> To view this discussion on the web visit
> >> https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com>.
> >> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Gremlin-users" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send
> > an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>
> > <mailto:gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/gremlin-users/F51985DD-7F40-45D7-BA18-FEC81E130099%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/F51985DD-7F40-45D7-BA18-FEC81E130099%40gmail.com>
> >
> <https://groups.google.com/d/msgid/gremlin-users/F51985DD-7F40-45D7-BA18-FEC81E130099%40gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gremlin-users/F51985DD-7F40-45D7-BA18-FEC81E130099%40gmail.com?utm_medium=email&utm_source=footer>>.
> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/7dc4ff30-0730-46ef-befc-ae8d2bf36e8d%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/7dc4ff30-0730-46ef-befc-ae8d2bf36e8d%40gmail.com>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-user...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/CAA-H43_wQ%2B_xABB-Q1n0NuEY9%2BYrsTdkyfFu7uKpbWnMBq3-gg%40mail.gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/CAA-H43_wQ%2B_xABB-Q1n0NuEY9%2BYrsTdkyfFu7uKpbWnMBq3-gg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Robert Dale

unread,
Dec 5, 2016, 10:06:58 AM12/5/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
Clearly we have different use cases.

You prefer your model to be that of the underlying graph (following that logic, you would use Hibernate to map to Table objects?) and I prefer using application domain models.

You prefer your query to return the underlying graph model and I prefer it to return any data.

You prefer your query to always return all properties and I prefer it to always return only selected properties.

You prefer your objects to be proxies to the underlying datastore (I think this blurs the lines between being a graph provider and gremlin client) and I prefer my objects to be detached with load/store being explicit.

In the end, it sounds like you want gremlin to be an object-graph mapper in the graph model and I prefer a layered approach where gremlin is a simple query language of which an object-graph mapper, in any domain model, could be built on top (like so many other query languages).

So I guess we'll just have to agree to disagree.


Robert Dale

>     <mailto:gremlin-users%2Bunsu...@googlegroups.com>
>     > <mailto:gremlin-users+unsub...@googlegroups.com

>     <mailto:gremlin-users%2Bunsu...@googlegroups.com>>.
>     > To view this discussion on the web visit
>     >
>     https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com
>     <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com>
>     >
>     <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer
>     <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer>>.
>     > For more options, visit https://groups.google.com/d/optout
>     <https://groups.google.com/d/optout>.
>
>     --
>     You received this message because you are subscribed to the Google
>     Groups "Gremlin-users" group.
>     To unsubscribe from this group and stop receiving emails from it,
>     send an email to gremlin-users+unsubscribe@googlegroups.com
>     <mailto:gremlin-users%2Bunsu...@googlegroups.com>.
>     To view this discussion on the web visit
>     https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com
>     <https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com>.
>     For more options, visit https://groups.google.com/d/optout
>     <https://groups.google.com/d/optout>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-users+unsubscribe@googlegroups.com
> <mailto:gremlin-users+unsub...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/CABed_4qE89f4oqZPQGjRXP8hn4kQpqVUiE%3DGq%2Bnvu_XfTQ_mWw%40mail.gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/CABed_4qE89f4oqZPQGjRXP8hn4kQpqVUiE%3DGq%2Bnvu_XfTQ_mWw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/31db7eef-046b-465f-13ea-0044a10da18c%40gmail.com.

Bryan Thompson

unread,
Dec 6, 2016, 10:18:54 AM12/6/16
to Gremlin-users, d...@tinkerpop.apache.org, Martyn Cutcher
A pattern we used for the CTC GOM (generic object model - much like tinker pop), was to allow dynamic casts of objects to registered interfaces. E.g.  

ObjectManager om = ...
om.registerIFace(MyDomainObject.class);

...

MyDomainObject obj = v.asClass(ifacename);

Martyn Cutcher took this much further with an alchemist project in which people authored domain application data models. Those data models were captured by annotations.  These annotations then supported code generation of various getters, setters, etc. for domain objects.  The object model then sat transparently behind those objects.

The GOM objects were known as GPO - generic persistent objects.  Each object was essentially a durable property map where some property values were simple properties and others were links.  Link sets collected the set of objects that pointed at a given object using the same link property.  You could then declare indices over those link sets.  This was all circa 1999.  The main difference between this and blueprints is that blueprints eagerly reifies edges as object instances.

Thanks,
Bryan

Robert Dale

>     > an email to gremlin-user...@googlegroups.com
>     <mailto:gremlin-users%2Bunsu...@googlegroups.com>
>     > <mailto:gremlin-users+unsub...@googlegroups.com
>     <mailto:gremlin-users%2Bunsu...@googlegroups.com>>.
>     > To view this discussion on the web visit
>     >
>     https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com
>     <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com>
>     >
>     <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer
>     <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer>>.
>     > For more options, visit https://groups.google.com/d/optout
>     <https://groups.google.com/d/optout>.
>
>     --
>     You received this message because you are subscribed to the Google
>     Groups "Gremlin-users" group.
>     To unsubscribe from this group and stop receiving emails from it,
>     send an email to gremlin-user...@googlegroups.com
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send

> <mailto:gremlin-users+unsub...@googlegroups.com>.
> To view this discussion on the web visit
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

pieter-gmail

unread,
Dec 6, 2016, 10:43:01 AM12/6/16
to gremli...@googlegroups.com, d...@tinkerpop.apache.org
Hi,

Now that we have established our disagreement lets continue...

"You prefer your model to be that of the underlying graph (following
that logic, you would use Hibernate to map to Table objects?) and I
prefer using application domain models."

When working with TinkerPop I certainly prefer that its model is that of
the underlying meta model i.e. having the property graph model as a
first class construct. For modeling
Cat/Dog/Person/Organization/Molecule... I also, like you use application
models as an abstraction on top of TinkerPop. Hibernate on top off
TinkerPop as opposed to JDBC if you will.

"You prefer your query to return the underlying graph model and I prefer
it to return any data."

True, however I am totally fine with g.V().this().that().values(...);
for the any data query.

"You prefer your query to always return all properties and I prefer it
to always return only selected properties."

This is partially true and indeed the default that I and I'd say the
TinkerPop test suites have assumed.
There are countless gremlins that return GraphTraversal<Vertex, Vertex>
that happily iterate the traversal accessing its properties assuming
that they are right there and not another db hit. It has bitten me
though with some fat vertices making it not such a sensible default
anymore. And when loading vertices just to create edges, then loading
the properties are somewhat silly.

"You prefer your objects to be proxies to the underlying datastore (I
think this blurs the lines between being a graph provider and gremlin
client) and I prefer my objects to be detached with load/store being
explicit."

True. My programming model assumes this to be the case and again so does
the TinkerPop test suite since day one. I'd argue it makes for eloquent
code. No need to worry about attaching and detaching and reattaching
with transaction semantics getting confused. You read and write within
the same transaction boundaries with the Vertex/Edge being bound to the
transaction. Very similar to you previous mail where you suggested
binding the interaction to a connection, except for the object we bind
to to is the transaction. Pretty much a one to one between transaction
and connection for most ACID databases.

"In the end, it sounds like you want gremlin to be an object-graph
mapper in the graph model and I prefer a layered approach where gremlin
is a simple query language of which an object-graph mapper, in any
domain model, could be built on top (like so many other query languages)."

Not so true. Object Graph Mappers will use TinkerPop as the graph layer
to model their domain on top off. TinkerPop responsibility is to model
its own model. The property graph model. One can think of
Vertex/Edge/Property as a baby hardcoded meta ORM but maybe its not a
useful analogy.

All that said I think we are not all that far apart as the way I read it
our primary disagreement is the properties.

Lets take GraphTraversal<Vertex, Vertex>.
If its a ReferencedVertex then every property access will be another
round trip. This seems unacceptable to me.
If it returns a Vertex with all its properties pre-loaded then its also
unacceptable. (Even if its the current default that I assume)

So perhaps what Marko suggested

g.withDetachment(…)
- Detach.reference // just ids
- Detach.reduced // ids, labels, keys, values (basically, what you
need for toString())
- Detach.rich // property data included
- Detached.full // edge data included (basically, StarGraph)

will makes us both happy?

Cheers
Pieter
> <pieter...@gmail.com <mailto:pieter...@gmail.com>
> > <mailto:pieter...@gmail.com
> > > v.id <http://v.id> <http://v.id>
> > > an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>
> > <mailto:gremlin-users%2Bunsu...@googlegroups.com
> <mailto:gremlin-users%252Buns...@googlegroups.com>>
> > > <mailto:gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>
> > <mailto:gremlin-users%2Bunsu...@googlegroups.com
> <mailto:gremlin-users%252Buns...@googlegroups.com>>>.
> > > To view this discussion on the web visit
> > >
> >
> https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com>
> >
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com>>
> > >
> >
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer>
> > <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gremlin-users/7CBD403D-4EC3-4B4B-AFF9-9A54B4D3C4EF%40gmail.com?utm_medium=email&utm_source=footer>>>.
> > > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>
> > <https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>>.
> >
> > --
> > You received this message because you are subscribed to the
> Google
> > Groups "Gremlin-users" group.
> > To unsubscribe from this group and stop receiving emails
> from it,
> > send an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>
> > <mailto:gremlin-users%2Bunsu...@googlegroups.com
> <mailto:gremlin-users%252Buns...@googlegroups.com>>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com>
> >
> <https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/79132fdd-f67f-5c3c-f8e3-87ab80f3c6f9%40gmail.com>>.
> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>
> > <https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>>.
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Gremlin-users" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send
> > an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>
> > <mailto:gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/gremlin-users/CABed_4qE89f4oqZPQGjRXP8hn4kQpqVUiE%3DGq%2Bnvu_XfTQ_mWw%40mail.gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/CABed_4qE89f4oqZPQGjRXP8hn4kQpqVUiE%3DGq%2Bnvu_XfTQ_mWw%40mail.gmail.com>
> >
> <https://groups.google.com/d/msgid/gremlin-users/CABed_4qE89f4oqZPQGjRXP8hn4kQpqVUiE%3DGq%2Bnvu_XfTQ_mWw%40mail.gmail.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/gremlin-users/CABed_4qE89f4oqZPQGjRXP8hn4kQpqVUiE%3DGq%2Bnvu_XfTQ_mWw%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-users%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/31db7eef-046b-465f-13ea-0044a10da18c%40gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/31db7eef-046b-465f-13ea-0044a10da18c%40gmail.com>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to gremlin-user...@googlegroups.com
> <mailto:gremlin-user...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/gremlin-users/CABed_4q456-_1jFk1qLWiNzNOTUxNZUhc3p6pCo1UYO6R6LBag%40mail.gmail.com
> <https://groups.google.com/d/msgid/gremlin-users/CABed_4q456-_1jFk1qLWiNzNOTUxNZUhc3p6pCo1UYO6R6LBag%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Reply all
Reply to author
Forward
0 new messages