Re: [TinkerPop] Problems using Titan with Rexster

[Floyd Hawkes]

What is the rest of the error if you don't mind.

[Stephen Mallette]

There's a long story to tell about POST/PUT/DELETE operations in
Rexster and the transactional semantics of the various graph
databases, that ends with me not noticing some silent failures in
Rexster integration tests and hence the problems you're seeing. I
don't know that the story has a happy ending, but I've pushed a fix to
github for the problems you are seeing. I think that if you do a
fresh pull and rebuild rexster, you should start seeing things working
properly.

Please note the "deviations" section on this page as those deviations
have an effect on what Rexster is capable of supporting with Titan:

https://github.com/thinkaurelius/titan/wiki/Blueprints-Interface

Those deviations in mind, one would still expect the REST API call to
POST an edge to work. Well...it will work, if you do this:

curl -X POST http://localhost:8182/graphs/titan/edges/

but not if you tried to do this:

curl -X POST http://localhost:8182/graphs/titan/edges/1234

where the latter is POSTing an edge with an identifier. The inclusion
of the identifier forces Rexster to call g.getEdge(id) which is not
supported by Titan and an error occurs.

DELETE of an edge has similar issues with Titan. Rexster calls
g.getEdge(id) and then uses it's return value to pass to
g.removeEdge(e). So a DELETE of an Edge won't work in Rexster. PUT
has the same limitation.

I've updated the Rexster page in the Titan wiki to reflect these limitations.

https://github.com/thinkaurelius/titan/wiki/Rexster-Graph-Server

Please let me know if there are other problems. We'll continue to try
to make all of this work together more seamlessly. Apologies for the
troubles.

Best regards,

Stephen

[Tyler Anderson]

thank you for such a thorough response, I was so sure it was titan causing the trouble i didn't rebuild rexster. It's always a mess when you have all of these different databases which implement varying levels of support for a third partys standard. I can confirm that a the code you pushed has resolved my issues.

[James Thornton]

Hi Stephen -

Have you given anymore thought to an RPC interface, using something like Thrift or Protocol Buffers?

- James

[Tyler Anderson]

I posted the same thing to OrientDB and thought i tacked the comment onto this convo as well. Thrift/ProtoBuffers/Avro are sorely lacking in the graph database world. Maybe one of the Titan guys might have some ideas on this as well because they're using Thrift to connect to their Cassandra storage layer i believe. I'd look into it myself, but my java experience is mostly using java classes in ruby/clojure. 

[Stephen Mallette]

James,

I've been quietly working on that. I've been kind of stealthy with it
because while the work I did on it was good enough for Rexster
Console, I wasn't sure if it was ultimately what I wanted it to be as
the partner in crime for the REST API. I've reworked the code for
this so many times now I've lost count. So...where is it now?

I'm finally starting to feel pretty good about what it's doing and how
it is designed and organized. I've settled on using MsgPack for
serialization. You send a string of Gremlin (with a few header bytes)
to that port and it sends back your results following a somewhat
similar pattern to the the JSON format that the REST API returns (i
had to make some adjustments there in that regard).

I've done some preliminary tests comparing RexPro to REST in EC2 and
the results were very encouraging, where RexPro was showing to be
clearly faster than the REST API. I'm still not quite sure it's ready
for people to start writing language bindings or basing their
applications on it, but the experiment is getting closer and closer to
something concrete and final. Maybe the release after this upcoming
one, will have a finalized specification for it.

Best regards,

Stephen

[Matthias Broecheler]

I don't think thrift/protocolbuffers/etc is the way to go when it comes to graph database access - at least not for distributed graph databases like Titan. Such an interface is most useful for primitive data access, i.e. get me a vertex or edge or the neighborhood. That's what Rexster already provides via REST.

However, the interesting queries over graph databases, and those where graph databases make a difference, are traversals. When you execute those locally against a distributed graph database, you are essentially pulling all those edges over the wire to conduct the traversal locally. That is very costly. In a way, that is what is happening when you run Titan locally and connect to a remote cluster (cassandra/hbase). With this setup, you are very limited in the kinds of traversals you can do (<200ms).

For better performance, you want to send the traversal to where the data is, that is, into the distributed graph database. This is what we are currently working on for Titan. You will be able to specify a gremlin query and then send that query into the Titan cluster (where titan is running locally with cassandra/hbase in the same jvm) where it will be processed and the result set is returned via a lightweight server using a compact protocol for low latency. The work Stephen is doing will be super useful here.

Now, you could argue that this is essentially the thrift/protocolbuffers/etc model, but I feel there is difference in philosophy. The former is for lower level data access. For distributed graph databases I suspect we will go toward sending fat traversal queries into the cluster. At least, that's what Titan will give you soon ;-)

--
Matthias Broecheler, PhD
http://www.matthiasb.com
E-Mail: m...@matthiasb.com

[Tyler Anderson]

I agree with what you are saying, and I actually completely understand the decisions you've made with Titan. Having said that, I do have some problems not the least of which is using graph databases outside of Java, even in Clojure its awkward so naturally in any non JVM language its even worse. At the moment Rexster is really the only way to use most graph databases outside java, and then you hit the other huge problem, compatibility. Every Blueprints "compatible" database has a different subset of functions, and there is nothing Rexster can do to bridge the gap. Honestly, I know that i'm not the only person who has been waiting for someone who has the time and vision to pull off what you are creating right now, in your shoes, with your goals i'd have done the same thing. Yet, because of the specific goals of your project, and their divergence from the Blueprints model, you fracture the ability of third parties to leverage the Tinkerpop stack as a compatibility layer. I'm currently writing software on top of Rexster, and the amount of tests,  code revisions, and patches i've had to write to get titan working are significant. I'm more than happy to support titan, and i'm having a hard time expressing both my personal opinion of titan which can be summed up as "FINALLY" with what i see as the larger problem of different database developers supporting wildly divergent subsets of the Blueprints graph model. On a final note,  from the perspective of someone building upon Rexster, what differences will your traversal queries provide over the Rexster gremlin plugin?

[James Thornton]

On Wednesday, August 1, 2012 4:35:13 PM UTC-5, Tyler Anderson wrote:

I agree with what you are saying, and I actually completely understand the decisions you've made with Titan. Having said that, I do have some problems not the least of which is using graph databases outside of Java, even in Clojure its awkward so naturally in any non JVM language its even worse.

Tyler -

What have you found awkward with using Titan in Clojure? 

I'm working on Bulbs/Clojure, and right now I'm focused on implementing the clients already in Bulbs/Python (i.e. Rexster and Neo4j Server), but Titan is next.

- James 

[Matthias Broecheler]

I agree with what you are saying, and I actually completely understand the decisions you've made with Titan. Having said that, I do have some problems not the least of which is using graph databases outside of Java, even in Clojure its awkward so naturally in any non JVM language its even worse.

Can you elaborate on this point?

 

At the moment Rexster is really the only way to use most graph databases outside java, and then you hit the other huge problem, compatibility. Every Blueprints "compatible" database has a different subset of functions, and there is nothing Rexster can do to bridge the gap. Honestly, I know that i'm not the only person who has been waiting for someone who has the time and vision to pull off what you are creating right now, in your shoes, with your goals i'd have done the same thing. Yet, because of the specific goals of your project, and their divergence from the Blueprints model, you fracture the ability of third parties to leverage the Tinkerpop stack as a compatibility layer. I'm currently writing software on top of Rexster, and the amount of tests,  code revisions, and patches i've had to write to get titan working are significant. I'm more than happy to support titan, and i'm having a hard time expressing both my personal opinion of titan which can be summed up as "FINALLY" with what i see as the larger problem of different database developers supporting wildly divergent subsets of the Blueprints graph model.

The two primary things that Titan does not support are:

1) Global Vertex/Edge iteration

2) Edge retrieval by id

Both are design decision, the former to avoid OOM and the latter in the interest of performance. The latter is hurting Rexster compatibility and I am currently designing a work around.

Are those two missing features the reason for your efforts or am I missing something?

 

On a final note,  from the perspective of someone building upon Rexster, what differences will your traversal queries provide over the Rexster gremlin plugin?

No difference. It basically allows you to send a gremlin query into the titan cluster and get the answer back. This will be much faster.

[Tyler Anderson]

On Wednesday, August 1, 2012 6:28:51 PM UTC-7, Matthias Broecheler wrote:

I agree with what you are saying, and I actually completely understand the decisions you've made with Titan. Having said that, I do have some problems not the least of which is using graph databases outside of Java, even in Clojure its awkward so naturally in any non JVM language its even worse.

Can you elaborate on this point?

       

      I was entirely wrong on this one, my statement was based upon my experiences with Clojure and Titan 2 months ago, my inexperience with the interop led me to do

      awkward things. Working with non JVM languages requires use of other projects, in my case Rexster that aren't specific to your database.

At the moment Rexster is really the only way to use most graph databases outside java, and then you hit the other huge problem, compatibility. Every Blueprints "compatible" database has a different subset of functions, and there is nothing Rexster can do to bridge the gap. Honestly, I know that i'm not the only person who has been waiting for someone who has the time and vision to pull off what you are creating right now, in your shoes, with your goals i'd have done the same thing. Yet, because of the specific goals of your project, and their divergence from the Blueprints model, you fracture the ability of third parties to leverage the Tinkerpop stack as a compatibility layer. I'm currently writing software on top of Rexster, and the amount of tests,  code revisions, and patches i've had to write to get titan working are significant. I'm more than happy to support titan, and i'm having a hard time expressing both my personal opinion of titan which can be summed up as "FINALLY" with what i see as the larger problem of different database developers supporting wildly divergent subsets of the Blueprints graph model.

The two primary things that Titan does not support are:

1) Global Vertex/Edge iteration

2) Edge retrieval by id

Both are design decision, the former to avoid OOM and the latter in the interest of performance. The latter is hurting Rexster compatibility and I am currently designing a work around.

Are those two missing features the reason for your efforts or am I missing something?

 

What i ended up doing was narrowing my own approach by cutting out the direct posting of edges with id's and any edge retrieval by id. 

On a final note,  from the perspective of someone building upon Rexster, what differences will your traversal queries provide over the Rexster gremlin plugin?

No difference. It basically allows you to send a gremlin query into the titan cluster and get the answer back. This will be much faster.

I'll look forward to that  

[Andras Gefferth]

I also get an "Internal Error" when I try to open the below url from a web-browser:

http://172.17.151.152:8182/graphs/test-titan/tp/gremlin

(If I replace "test-titan" with the name of another configured graph I get a meaningful JSON error message.)

My test-titan graph in rexster.xml looks like this:

     <graph>

          <graph-name>test-titan</graph-name>

          <graph-type>com.thinkaurelius.titan.tinkerpop.rexster.TitanGraphConfiguration</graph-type>

          <graph-read-only>false</graph-read-only>

          <graph-location>/root/data/test/titan</graph-location>

          <properties>

            <storage.backend>local</storage.backend>

            <storage.directory>/root/data/test/titan</storage.directory>

            <buffer-size>100</buffer-size>

          </properties>

          <extensions>

            <allows>

              <allow>tp:gremlin</allow>

            </allows>

          </extensions>

        </graph>            

I can access and manipulate the graph with the doghouse gremlin interface, but when I try to connect with bulbs or thunderdome I get error messages.

I use the latest 2.3.0-SNAPSHOT of Rexster and copied the files from titan 0.2.0 /lib to /ext/titan under the Rexster directory.

Is this the expected behavior, is this a bug, or am I missing something maybe?

thanks

[Stephen Mallette]

Does the URL you referenced return meaningful JSON when you first
start Rexster (prior to accessing with bulbs/thunderdome/dog house)?

Stephen

> --
> You received this message because you are subscribed to the Google Groups
> "Gremlin-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to gremlin-user...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Andras Gefferth]

On Tuesday, February 5, 2013 1:49:21 PM UTC+1, Stephen Mallette wrote:

Does the URL you referenced return meaningful JSON when you first
start Rexster (prior to accessing with bulbs/thunderdome/dog house)?

 

No, I get the same message.

Andras

[Stephen Mallette]

I find it strange that you can:

> access and manipulate the graph with the doghouse gremlin interface, but when I try to connect with bulbs or thunderdome I get error messages.

I do assume that you mean your Titan graph there. I'd guess that
something was wrong in the <http> section of your rexster.xml but you
say that you can access other configured graphs without issue. Do you
find any error messages in the terminal you are running Rexster from
as you make these requests? what about error messages at startup
(titan usually is pretty explicit in its output if there are problems
at start of rexster)? forgetting the gremlin extension for a moment,
does the titan graph work if you access it via the standard REST API
at startup (like
http://localhost:8182/graphs/test-titan/vertices/{some-id})?

[Andras Gefferth]

On Tuesday, February 5, 2013 2:23:50 PM UTC+1, Stephen Mallette wrote:

I find it strange that you can:

> access and manipulate the graph with the doghouse gremlin interface, but when I try to connect with bulbs or thunderdome I get error messages.

I do assume that you mean your Titan graph there.

Yes 

 I'd guess that something was wrong in the <http> section of your rexster.xml but you
say that you can access other configured graphs without issue.

Yes 

Do you  find any error messages in the terminal you are running Rexster from
as you make these requests?

When I try to access the above url with a web browser, I see the following message, but only for the first time:

 [WARN] GraphResource - The [tp:gremlin+*] extension raised an error response.

When connecting with bulbs, I get:

[ERROR] GremlinExtension - Gremlin Extension: javax.script.ScriptException: groovy.lang.MissingMethodException: No signature of method: groovy.lang.MissingMethodException.idx() is applicable for argument types: () values: []

Possible solutions: is(java.lang.Object), find(), any(), _(groovy.lang.Closure), with(groovy.lang.Closure), use([Ljava.lang.Object;)

javax.script.ScriptException: javax.script.ScriptException: groovy.lang.MissingMethodException: No signature of method: groovy.lang.MissingMethodException.idx() is applicable for argument types: () values: []

...

LOTS OF "at.org....." LINES HERE TOGETHER WITH THE PREVIOUS MESSAGE REPEATED SEVERAL TIMES

what about error messages at startup (titan usually is pretty explicit in its output if there are problems 

at start of rexster)?

None.

 

 

forgetting the gremlin extension for a moment,
does the titan graph work if you access it via the standard REST API
at startup (like
http://localhost:8182/graphs/test-titan/vertices/{some-id})?

Yes, it works at startup, and also after failed bulbs/thunderdome attempts

It works both with and without supplying "some-id"

 

Failed attempts do not seem to influence what is working and what is not.

Andras

[Stephen Mallette]

Given your error, it looks like a compatibility issue with bulbs/titan:

No signature of method: groovy.lang.MissingMethodException.idx()

Titan doesn't support IndexableGraph. It only implements
KeyIndexableGraph. Maybe James Thornton has a comment or two on this
situation. Can't say what thunderdome is doing unless you are issuing
a g.idx style gremlin query in which case the problem is the same.

Stephen

[Andras Gefferth]

On Tuesday, February 5, 2013 3:55:21 PM UTC+1, Stephen Mallette wrote:

Given your error, it looks like a compatibility issue with bulbs/titan:

 

It can be.

Although I have the issue with the web browser and thunderdome as well.

This may, however, be a different issue, since in these latter cases I receive no error messages in the console where rexster is started.

[Andras Gefferth]

I can now confirm that there are two separate issues with titan-rexster.

I tried the same setup with rexster 2.2.0 instead of 2.3.0. (Same rexster.xml file, same titan java files)

1, Now accessing http://host:8182/graphs/test-titan/tp/gremlin gives a valid JSON error message and I can connect with thunderdome.

This indicates a bug in Rexster 2.3.0

2, bulbs is still not working.

This indicates incompatibility.

Andras

[Stephen Mallette]

Thanks for providing more information Andras.

I think I already addressed the Bulbs issue. Try as I might, I can't
seem to recreate the issue. Can anyone working on Thunderdome
recreate the problem here? Are there compatibility issues at play
among Rexster/Thunderdome/Titan?

Thanks,

Stephen

[Jonathan Haddad]

We rolled out Rexster 2.2 to production this week, along with the latest version of thunderdome in pip.  We haven't tried the latest Rexter master with Thunderdome yet.

If you can post a code snippet (a gist, preferably) of what you're doing maybe we can be of more assistance.

Jon

[Andras Gefferth]

Hi All,

sorry for not responding, I was on vacation for a few days.

To me it seems that the problem is not Thunderdome specific, since I have the issue from a web browser as well.

If you have some time to look at the issue I would suggest switching from Rexster 2.2. to 2.3 and see if it works there?

The rexster version I used is commit 827d55f4df03431f54101a3a1fedca3f0132f5bb.

I used the .jars from titan 0.2.0

The initial graph I created as described here: https://github.com/thinkaurelius/titan/wiki/Getting-Started (g = TitanFactory.open('/tmp/titan') )

My rexster.xml is attached.

The url which gives me the internal error is:

http://localhost:8182/graphs/test-titan/tp/gremlin

Andras

[Stephen Mallette]

Andras, something just occurred to me. I had to read back through the
thread here and I think we need to clarify something. When you say:

>The url which gives me the internal error is:
>http://localhost:8182/graphs/test-titan/tp/gremlin

Are you specifically referring to an "Internal Server
Error"...basically http status 500?

Stephen

[James Thornton]

Andras -

Just saw this -- are you importing bulbs.titan or bulbs.rexster?

Titan indexing differs from traditional Blueprints implementations so a few months ago I added a bulbs.titan adapter. 

Bulbs-Titan inherits from Bulbs Rexster so most of the interface remains the same -- there are some low-level changes to titan.client.py:

https://github.com/espeed/bulbs/blob/master/bulbs/titan/client.py

...but you don't typically need to call Client directly.

The only high-level changes were to titan/index.py:

https://github.com/espeed/bulbs/blob/master/bulbs/titan/index.py

Here's a Bulbs/Titan example: https://gist.github.com/espeed/3938820

- James

[Andras Gefferth]

Yes.

The output of wget is:

Connecting to 172.17.151.152:8182... connected.

HTTP request sent, awaiting response... 500 Internal Error

2013-02-14 17:25:18 ERROR 500: Internal Error.

[Stephen Mallette]

ok...I get the same:

curl -v http://localhost:8182/graphs/tinkergraph/tp/gremlin

* About to connect() to localhost port 8182 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8182 (#0)
> GET /graphs/tinkergraph/tp/gremlin HTTP/1.1
> User-Agent: curl/7.21.6 (x86_64-pc-linux-gnu) libcurl/7.21.6 OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
> Host: localhost:8182
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< server: grizzly/2.2.16
< Access-Control-Allow-Origin: *
< Date: Thu, 14 Feb 2013 16:39:43 GMT
< Connection: close
< Transfer-Encoding: chunked
<

That's an expected response for that URI if there's no script supplied
for Rexster to run. The Rexster extension needs to be called in this
way (at least this is a common way to call it):

curl -v "http://localhost:8182/graphs/tinkergraph/tp/gremlin?script=g.v(1)"

I suppose in the INTERNAL_SERVER_ERROR isn't so good an error
response. BAD_REQUEST would be better. I'll change the status code
to make it more clear, though the message in the body of the response
(which isn't rendering with wget) would tell you that a script for
rexster to process is not present.

Stephen

[Andras Gefferth]

I tried both

With bulbs.rexster I receive the "Gremlin Extension: javax.script.ScriptException: groovy.lang.MissingMethodException ... " error in the terminal window where rexster was started.

It occurs when I try to execute "g=Graph(config)"

Python gives the error message:

SystemError: ({'status': '500', 'transfer-encoding': 'chunked', 'server': 'grizzly/2.2.18', 'connection': 'close', 'date': 'Thu, 14 Feb 2013 16:35:52 GMT', 'content-type': 'text/html;charset=ISO-8859-1'}, '<html><body><h1>Internal Error</h1></body></html>')

With bulbs.titan I see no error message in the terminal.

The error occurs at a later stage when I execute "g.V"

I get the same 500-error message in Python.

This is with Rexster 2.3.0.

---

If I use Rexster 2.2.0:

using bulbs.rexster the symptoms are very similar to the above, only in python I receive a much longer error message.

using bulbs.titan the error happens during a g.vertices.delete operation (not necessarily the first one)

and the error message is:

SystemError: ({'status': '500', 'transfer-encoding': 'chunked', 'server': 'grizzly/2.2.18', 'connection': 'close', 'date': 'Thu, 14 Feb 2013 16:49:02 GMT', 'access-control-allow-origin': '*', 'content-type': 'application/json;charset=UTF-8'}, '{"message":"Could not commit transaction due to exception during persistence","error":"Could not commit transaction due to exception during persistence"}')

---

with tinkergraph and neo4j I can use both 2.2.0 and 2.3.0 without problems

I hope this helps
This issue is not critical to me at the moment, but would be good to understand what's going on.

Andras

[Andras Gefferth]

Unfortunately I get the same error if I try with a script.

2.2.0 works, 2.3.0 gives error

Andras

[Alex Buchanan]

Did anyone follow up on this? With Titan 0.3.2 and Rexster 2.4 I still get a 500 "Internal Error" from curl http://localhost:8182/graphs/titanexample

[Alex Buchanan]

Oh, sorry, I didn't realize this was a gremlin-users group (not a rexster or titan group) so maybe that question isn't appropriate here.

I think I found my answer here: https://groups.google.com/d/msg/aureliusgraphs/4RlgI9qQUdg/7FS72_vKSDIJ

[James Thornton]

The default Titan graph name is now "graph". Is the graph "titanexample" created and set in the rexster.xml file? 

- James