[bulbs] Not using Titan indices?

184 views
Skip to first unread message

Darren Govoni

unread,
Dec 20, 2013, 9:24:17 AM12/20/13
to aureliu...@googlegroups.com
Hi,
  I try to do a simple lookup in bulbs and titan.

vertices = g.vertices.index.lookup(name="Military")

In the titan console it reads:

230375932 [Grizzly(2)] WARN  com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx  - Query requires iterating over all vertices [(v[36028797018964082]=Military)]. For better performance, use indexes

And the lookup takes more than a few seconds. Like 10 seconds (way too long for less than 100,000 verticies).

What's wrong here?

thank you!

Marko Rodriguez

unread,
Dec 20, 2013, 11:45:56 AM12/20/13
to aureliu...@googlegroups.com
Hello,

You need to create an index on "name" as yo are doing a linear scan over all vertices to find the vertex with property "name=military."

Please read:

Good luck,
Marko.
--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Darren Govoni

unread,
Dec 20, 2013, 11:59:25 AM12/20/13
to aureliu...@googlegroups.com
Thank you sir. It can be tricky translating from the bulbs examples which might assume some prior config in place.

Darren Govoni

unread,
Dec 20, 2013, 12:51:57 PM12/20/13
to aureliu...@googlegroups.com
So this appears to be Java code. I don't have a Java application.

Is there some RESTful way to create an index (better if bulbs supported it directly. )?

is there any out-of-the-box indexes I can write my vertex/edge properties against to avoid this extra step?


On Friday, December 20, 2013 11:45:56 AM UTC-5, Marko Rodriguez wrote:

Marko Rodriguez

unread,
Dec 20, 2013, 12:54:23 PM12/20/13
to aureliu...@googlegroups.com
Hello,

You can just use Gremlin to create indices. Gremlin allows "Java code" to be submitted to it.

g.makeKey("name").indexed(Vertex.class).make()
g.commit()
g.addVertex(["name","marko"])
g.commit()
g.V("name","marko").toList()

HTH,
Marko.

Darren Govoni

unread,
Dec 21, 2013, 8:32:53 AM12/21/13
to aureliu...@googlegroups.com
Excellent! Thank you sir.

Darren Govoni

unread,
Dec 21, 2013, 8:54:04 AM12/21/13
to aureliu...@googlegroups.com
FYI.

I tried this:

>>> from bulbs.titan import Graph
>>> g=Graph()
>>> a=g.client.gremlin('g.makeKey("name").indexed(Vertex.class).make()')

Got errors....

I'm using titan-cassandra-0.3.2


78788575 [Grizzly(7)] ERROR com.tinkerpop.rexster.gremlin.GremlinExtension  - Gremlin Extension: javax.script.ScriptException: groovy.lang.MissingMethodException: No signature of method: groovy.lang.MissingMethodException.makeKey() is applicable for argument types: () values: []
Possible solutions: every(), any()
javax.script.ScriptException: javax.script.ScriptException: groovy.lang.MissingMethodException: No signature of method: groovy.lang.MissingMethodException.makeKey() is applicable for argument types: () values: []
Possible solutions: every(), any()
at com.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:110)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
at com.tinkerpop.rexster.gremlin.GremlinExtension.tryExecuteGremlinScript(GremlinExtension.java:253)
at com.tinkerpop.rexster.gremlin.GremlinExtension.evaluatePostOnGraph(GremlinExtension.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.tinkerpop.rexster.AbstractSubResource.invokeExtension(AbstractSubResource.java:321)
at com.tinkerpop.rexster.AbstractSubResource.invokeExtension(AbstractSubResource.java:229)
at com.tinkerpop.rexster.GraphResource.executeGraphExtension(GraphResource.java:269)
at com.tinkerpop.rexster.GraphResource.postGraphExtension(GraphResource.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
at org.glassfish.grizzly.servlet.FilterChainImpl.doFilter(FilterChainImpl.java:147)
at org.glassfish.grizzly.servlet.FilterChainImpl.invokeFilterChain(FilterChainImpl.java:106)
at org.glassfish.grizzly.servlet.ServletHandler.doServletService(ServletHandler.java:252)
at org.glassfish.grizzly.servlet.ServletHandler.service(ServletHandler.java:188)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:164)
at org.glassfish.grizzly.http.server.HttpHandlerChain.service(HttpHandlerChain.java:196)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:164)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:175)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:265)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:200)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:134)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:78)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:815)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
at org.glassfish.grizzly.strategies.LeaderFollowerNIOStrategy.executeIoEvent(LeaderFollowerNIOStrategy.java:102)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.executeIoEvent(AbstractIOStrategy.java:88)
at org.glassfish.grizzly.nio.SelectorRunner.iterateKeyEvents(SelectorRunner.java:398)
at org.glassfish.grizzly.nio.SelectorRunner.iterateKeys(SelectorRunner.java:368)
at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:334)
at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:264)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:567)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:547)
at java.lang.Thread.run(Thread.java:744)
Caused by: javax.script.ScriptException: groovy.lang.MissingMethodException: No signature of method: groovy.lang.MissingMethodException.makeKey() is applicable for argument types: () values: []
Possible solutions: every(), any()
at com.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:237)
at com.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:106)
... 56 more
Caused by: groovy.lang.MissingMethodException: No signature of method: groovy.lang.MissingMethodException.makeKey() is applicable for argument types: () values: []
Possible solutions: every(), any()
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:55)
at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at Script2.run(Script2.groovy:1)
at com.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:235)
... 57 more
78788584 [Grizzly(7)] WARN  com.tinkerpop.rexster.GraphResource  - The [tp:gremlin+*] extension raised an error response.
Dec 21, 2013 8:51:45 AM com.sun.jersey.spi.container.ContainerResponse logException
SEVERE: Mapped exception to response: 500 (Internal Server Error)
javax.ws.rs.WebApplicationException
at com.tinkerpop.rexster.GraphResource.executeGraphExtension(GraphResource.java:301)
at com.tinkerpop.rexster.GraphResource.postGraphExtension(GraphResource.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
at org.glassfish.grizzly.servlet.FilterChainImpl.doFilter(FilterChainImpl.java:147)
at org.glassfish.grizzly.servlet.FilterChainImpl.invokeFilterChain(FilterChainImpl.java:106)
at org.glassfish.grizzly.servlet.ServletHandler.doServletService(ServletHandler.java:252)
at org.glassfish.grizzly.servlet.ServletHandler.service(ServletHandler.java:188)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:164)
at org.glassfish.grizzly.http.server.HttpHandlerChain.service(HttpHandlerChain.java:196)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:164)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:175)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:265)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:200)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:134)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:78)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:815)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
at org.glassfish.grizzly.strategies.LeaderFollowerNIOStrategy.executeIoEvent(LeaderFollowerNIOStrategy.java:102)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.executeIoEvent(AbstractIOStrategy.java:88)
at org.glassfish.grizzly.nio.SelectorRunner.iterateKeyEvents(SelectorRunner.java:398)
at org.glassfish.grizzly.nio.SelectorRunner.iterateKeys(SelectorRunner.java:368)
at org.glassfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:334)
at org.glassfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:264)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:567)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:547)
at java.lang.Thread.run(Thread.java:744)

Darren Govoni

unread,
Dec 21, 2013, 9:05:20 AM12/21/13
to aureliu...@googlegroups.com
Gonna try the latest titan...

Darren Govoni

unread,
Dec 21, 2013, 9:38:22 AM12/21/13
to aureliu...@googlegroups.com
Doesn't work in 0.4 either.....

This seems to be much more difficult getting started than I expected.

Maybe Titan could benefit from some kind of out-of-the-box metamodel for vertices and edges that are already indexed.

2 days spent trying to retrieve small amount of data in under 10 seconds...

Help appreciated.

Marko Rodriguez

unread,
Dec 21, 2013, 10:48:42 AM12/21/13
to aureliu...@googlegroups.com
Hi,

I'm not an expert on Bulbs . You might want to ping James Thornton about it. But, here is how to make a simple graph.

1. Drop into Gremlin.
2. Open a new TitanGraph.
3. Add vertices.
4. Enjoy the riches.

~/software/aurelius/titan$ bin/gremlin.sh

         \,,,/
         (o o)
-----oOOo-(_)-oOOo-----
gremlin> g = TitanFactory.open('conf/titan-cassandra-embedded.properties')
==>titangraph[embeddedcassandra:null]
gremlin> g.makeKey('name').indexed(Vertex.class).make()
Need to specify a datatype
Display stack trace? [yN]
gremlin> g.makeKey('name').indexed(Vertex.class).dataType(String.class).make()
==>v[74]
gremlin> g.commit()
==>null
gremlin> g.addVertex(['name':'marko'])
==>v[4]
gremlin> g.commit()
==>null
gremlin> g.V('name','marko')
==>v[4]
gremlin> "tada!"
==>tada!
gremlin>

Realize, when I typed in the code I gave you, I got an exception saying "You need to specify the datatype" so I added dataType(String.class).

Also, if you want a play data set to work with, please see:

Good luck,
Marko.

Darren Govoni

unread,
Dec 21, 2013, 2:38:24 PM12/21/13
to aureliu...@googlegroups.com
Thanks again Marko. I will try this. I did post to the gremlin list but didn't get a response so I thought to try here. Appreciate it.

Darren Govoni

unread,
Dec 22, 2013, 10:25:46 AM12/22/13
to aureliu...@googlegroups.com
I guess what I don't understand about this aspect of Titan is why I have to manually tell it to index certain properties.
Mongo seems to do it automatically on its document properties and its much easier to use. Just some constructive feedback.

Marko Rodriguez

unread,
Dec 22, 2013, 12:13:38 PM12/22/13
to aureliu...@googlegroups.com
Hello,

If you index every possible key you will have lots of indices. Like in MySQL, you create an index when you don't want to do a linear scan. This can be determined for you as it really depends on your data model and how you are querying it.

Next, I doubt MongoDB would automatically index everything. This would use a huge disk footprint.

Take care,
Marko.
Reply all
Reply to author
Forward
0 new messages