Multiple Graphs in Gremlin Server/Rexster

[Stephen Mallette]

A quick survey: Do you serve multiple graphs in Gremlin Server/Rexster AND do multi-graph queries in a single script request?  

I'll assume that silence means that you never do such a thing, otherwise please reply with a "Yes".  If you care to share added details about your use case in this situation, then please feel free.

[Jonathan Kelsey]

Yes and in the future Yes.

[Stephen Mallette]

Interesting - so I've had this thread out there for about a week and there's been one reply.  I'd expected more.  

Jonathan, as the only representative for this feature, I guess I have to lean on your use case a bit to understand what you're doing.  There are several problems with this feature, not the least of which is transaction management.  What do you do with a multi-graph query where a transaction against one of the graphs fails?  That seems like a real problem.  For example, assuming you sent:

g1.addV()

g2.addV()

g3.addV()

and g1 and g3 succeeded , but g2 failed - does your use case not have that issue somehow?

-------------------------
The information in this message is private and confidential and may be legally privileged. If you have received this message in error, please notify us and remove it from your system. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.

For more information:
     Twitter: @semblent1
     LinkedIn: https://www.linkedin.com/company/semblent
     The Internet: www.semblent.com




-------------------------
The information in this message is private and confidential and may be legally privileged. If you have received this message in error, please notify us and remove it from your system. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.

For more information:
     Twitter: @semblent1
     LinkedIn: https://www.linkedin.com/company/semblent
     The Internet: www.semblent.com

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/d1caf887-b59b-4074-82b3-d74ca0091828%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jonathan Kelsey]

I'm going to investigate using the same 'read' traversal in multiple graphs. I hadn't considered writing any data that way. 

[Stephen Mallette]

My guess would be that reads would not be an issue really.  The concern would be more with writes, where one write succeeds and others fail.  There is nothing coordinating those transactions across databases.  So while your read-only use case is probably safe, we leave open a might large hole for someone to shoot themselves in the foot if they do multi-database mutations.  That part kinda bugs me........

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/bb7946d5-9f7e-47e0-b2e0-48937c7bbe72%40googlegroups.com.

[Todd Leo]

Hi Stephen,

What is your use case to connect multiple Gremlin Server or, multiple graphs in one Gremlin Server(is this possible?)?

I’m having troubles with the former when trying to connect to different tables in HBase, as I use separate gremlin-server.yaml with separate port and tablename, and start Gremlin Server with gremlin-server.sh my-new-gremlin-server.yaml. Then in gremlin shell, connect to the Gremlin Server by specifying a separate remote.yaml with the new port to connect. However, it appears that I have connected to the original graph with nodes and edges data which I created before. Does anyone tried this way, or there exists other ways to connect to multiple graphs simultaneously?

---

BR,
Todd Leo

​

[Stephen Mallette]

You can have as many graphs served from Gremlin Server as you'd like (or, more aptly said, as many as your machine running gremlin server can support).  You just add additional entries to the "graphs" key in the server config file.  

as for your problem with running multiple Gremlin Servers - not sure what use case you would have to do that on the same machine - you didn't mention the step where you provide a new graph properties file to the "graphs" key in your server config file.  it is that file that controls what graph database you would connect to.

--

You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/b27be872-ae1f-4f35-b6eb-9211482ff9e7%40googlegroups.com.

[SLiZn Liu]

Hi Stephen,

Good to know multiple graphs can be server from a single Gremlin Server. When you say,

…add additional entries to the “graphs” key in the server config file.

did you mean add the following entries in gremlin-server.yaml:

graphs: {                                           
  graph: conf/gremlin-server/titan-hbase-TABLE-1.properties,

  graph: conf/gremlin-server/titan-hbase-TABLE-2.properties,
  ...
  graph: conf/gremlin-server/titan-hbase-TABLE-n.properties
}

and, how to specify the graph I’d like to connect afterwards?

Regards to the multiple Gremlin Servers I mentioned, it should be no longer considered now that multiple graphs can be hosted by a single Gremlin Server.

---

BR,
Todd Leo

​

You received this message because you are subscribed to a topic in the Google Groups "Gremlin-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gremlin-users/6l9-z5g7FYI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H43997%3DdQC30t__T7z3y4991iEG9b_%2B8veFB0oDkBHQAasw%40mail.gmail.com.

[Jason Plurad]

Here's one approach.

In conf/gremlin-server/gremlin-server.yaml, you would need each graph to be named uniquely under graphs. Keep in mind that if you're using the same HBase cluster for all of these graphs, you should have a unique storage.hbase.table defined in each properties file (default table name is 'titan')

graphs: {                                          
  graph1: conf/gremlin-server/titan-hbase-TABLE-1.properties,
  graph2: conf/gremlin-server/titan-hbase-TABLE-2.properties,
  ...
  graphn: conf/gremlin-server/titan-hbase-TABLE-n.properties
}

Then you may want to bind a unique graph traversal source for each graph, so update scripts/empty-sample.groovy also

// define the default TraversalSource to bind queries to.
g1 = graph1.traversal();
g2 = graph2.traversal();
gn = graphn.traversal();

[SLiZn Liu]

Very well Jason, didn’t realize graph instance name corresponds to the entrie of conf files in gremlin-server.yaml. Is this documented anywhere? It might also helpful to others.

​

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/be829684-648f-4797-8c3f-d5c6100da278%40googlegroups.com.

[SLiZn Liu]

No the issue is not solved yet. I added two graphs in gremlin-server.yaml, with each corresponding titan-hbase.properties specifying different HBase tables(“titan” and “new”). The problem remains, as I described in the first E-mail I sent:

it appears that I have connected to the original graph with nodes and edges data which I created before.

Here is the code I’ve used in gremlin shell:

gremlin> :remote connect tinkerpop.server conf/remote.yaml
==>Connected - localhost/127.0.0.1:8182

gremlin> :> g=graph1.traversal();  g.V().values('name')
==>david
gremlin> :> g2=graph2.traversal();  g2.V().values('name')
==>david // This is odd because graph2 is supposed empty

gremlin> :> graph1.addVertex("name", "todd") // To verify, let's add a new node to graph1
==>v[4320]

gremlin> :> graph1.tx().commit()
==>null
gremlin> :> g=graph1.traversal();  g.V().values('name')
==>david
==>todd
gremlin> :> g=graph2.traversal();  g.V().values('name')
==>david
==>todd // Clearly, this node is present for both graphs!!

gremlin-server.yaml

graphs: {                                               
  graph1: conf/gremlin-server/titan-hbase.properties,   
  graph2: conf/gremlin-server/titan-hbase-new.properties
}                                                       

# Other parts of this file remains unchanged

titan-hbase.properties

gremlin.graph=com.thinkaurelius.titan.core.TitanFactory
storage.backend=hbase                                  
storage.hostname=127.0.0.1                             
storage.hbase.tablename=titan

titan-hbase-new.properties

gremlin.graph=com.thinkaurelius.titan.core.TitanFactory
storage.backend=hbase                                  
storage.hostname=127.0.0.1                             
storage.hbase.tablename=new

remote.yaml is also unchanged.

---

BR,
Todd Leo

​

[Stephen Mallette]

I'm not sure what's wrong, but I just tried hosting multiple TinkerGraph implementations and that works fine. I don't know titan+hbase so well.  Maybe Jason can spot where things are messed up.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAD_PBkTw6Ao-SwmUefxkdLCqmo13swtxbgyFKrJnkysyTDfDhQ%40mail.gmail.com.

[Jason Plurad]

The key in the properties files should be storage.hbase.table

http://s3.thinkaurelius.com/docs/titan/1.0.0/titan-config-ref.html#_storage_hbase

[SLiZn Liu]

Hi Jason,

It works like a charm as I corrected the key. I copied them elsewhere and didn't notice the difference. Really appreciate for your and Stephen's help!

---

BR,

Todd Leo

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/32796ea7-74f3-4ac4-aa46-20766adf69e8%40googlegroups.com.

[SLiZn Liu]

Could you guys also please checkout another problem of mine, when starting Gremlin Server on another set of machines? 

https://groups.google.com/d/msg/aureliusgraphs/PJW64yyziqE/iIpDV8E-BQAJ

On Wed, Dec 2, 2015 at 11:18 PM Jason Plurad <plu...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/32796ea7-74f3-4ac4-aa46-20766adf69e8%40googlegroups.com.

[ma...@oneilinteractive.com]

I am just seeing the Getting Started Tutorial and I think that it is great. One thing that I think is missing are details on how to handle remote queries for multiple graphs. In 2.x you simply passed the graph as apart of the URI and provided your script (g.V()..., where g was the graph instance). From what I understand now, each graph defined in the yaml config is set to a variable, however, that variable is an instance of the graph and not the traversal (it seems that the g variable is a traversal by default). Is there a setting or a common way to have the traversal available for each defined graph by default or do i have to do something like:

g2_traversal = g2.traversal();
or
g2.traversal().methodOnlyAvailableOnATraversal(...);

[Stephen Mallette]

I'm not so sure I can pack too much more into our "Getting Started" tutorial. :)  You initialize "g" (your TraversalSource) via init script that launches when Gremlin Server starts.  You can read the details here in the reference docs:

http://tinkerpop.apache.org/docs/3.1.0-incubating/#connecting-via-console

We probably need a Gremlin Server tutorial - "coming soon" i suppose.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/a51534d6-2c1a-4408-8d90-e50d965ac350%40googlegroups.com.

[ma...@oneilinteractive.com]

I see. Thank you very much. I read that section and glossed over those details. 

For those who may not know, there is an init script that is run when you start the server that defines what 'g' is. What I did was add my defined graph as a 'g' into that script

conf/gremlin-server.yam

graphs: {

  graph: conf/tinkergraph-empty.properties,

  mark: conf/tinkergraph-mark.properties,

}

scripts/empty-sample.groovy

globals << [g : graph.traversal()]

globals << [markg: mark.traversal()]

Now I can send multiple scripts to the server g.v(); markg.v(); etc and mutate them individually. 

It may be helpful to explain this detail a little clearer in the documentation. 

Thanks again.