Configuring Gremlin Server apache-gremlin-server-3.0.1 Incubating with Titan 1.0.0 Hadoop 1

[anton...@gmail.com]

I am new to the Gremlin Server and I am trying to setup Gremlin Server version https://www.apache.org/dist/incubator/tinkerpop/3.0.1-incubating/apache-gremlin-server-3.0.1-incubating-bin.zip with Titan Graph 1.0.0 Hadoop 1

When I issue the following command to get the dependencies

bin/gremlin-server.sh -i com.thinkaurelius.titan titan-all 1.0.0

I am getting the following error

$  bin/gremlin-server.sh -i com.thinkaurelius.titan titan-all 1.0.0

Could not install the dependency: Error grabbing Grapes -- [unresolved dependency: org.apache.giraph#giraph-core;1.0.0: not found]

java.lang.RuntimeException: Error grabbing Grapes -- [unresolved dependency: org.apache.giraph#giraph-core;1.0.0: not found]

        at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)

        at groovy.grape.GrapeIvy.getDependencies(GrapeIvy.groovy:418)

        at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)

        at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:563)

        at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)

        at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:528)

        at groovy.grape.Grape.resolve(Grape.java:198)

        at org.codehaus.groovy.vmplugin.v7.IndyInterface.selectMethod(IndyInterface.java:215)

        at org.apache.tinkerpop.gremlin.groovy.util.DependencyGrabber.copyDependenciesToPath(DependencyGrabber.groovy:86)

        at org.apache.tinkerpop.gremlin.server.util.GremlinServerInstall.main(GremlinServerInstall.java:38)

how do I get around this error?

Thanks

Anton

[Jason Plurad]

The easiest thing to do would be to pull down the Titan 1.0.0 Hadoop 1 zip from the Titan Downloads page. Here's a direct link:

http://s3.thinkaurelius.com/downloads/titan/titan-1.0.0-hadoop1.zip

Titan 1.0.0 uses TinkerPop 3.0.1, and the distribution ships with gremlin-server.sh and config files tailored for Titan out of the box.

Otherwise, if you don't want to do that, you'd have to configure a $HOME/.groovy/grapeConfig.xml with a remote repository that has the Giraph 1.0.0 dependency because it is no longer on Maven Central.

<ibiblio name="hyracks" root="http://obelix.ics.uci.edu/nexus/content/groups/hyracks-public-releases/" m2compatible="true"/>

Here's where the TinkerPop3 build pulls it in https://github.com/apache/incubator-tinkerpop/blob/3.0.1-incubating/hadoop-gremlin/pom.xml#L213

[Stephen Mallette]

Note that I've added docs back to Titan for manual installation and it makes note of that grapeConfig.xml  issue:

https://github.com/thinkaurelius/titan/commit/00c187ede45e5d6e80d779edcedac9d0399b3b31

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/aureliusgraphs/c11ab83d-a6c9-4a38-9ab4-2b38466de765%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[anton...@gmail.com]

Thanks Jason, Stephen,

I have already played around with the single instance titan.sh which uses the backend as casandra and elastic search.

What I am trying to do is to make a distributed deployment using multiple gremlin servers having HBase Cluster as storage layer and Elastic Search Cluster as Index layer.

I already have a two node HBase cluster and two node Elastic Search cluster.

I was able to configure Titan to work of these clusters using the gremlin shell but facing some issue or the other with the gremlin server configuration.I spent some time configuring with Rexster only to realize that Titan 1.0.0 does not support Rexter anymore.

So following your instructions, namely creating a file called grapeConfig.xml and placing it under the current users home directory under ~/.groovy and copying the contents of http://tinkerpop.incubator.apache.org/docs/3.0.1-incubating/#gremlin-applications into the grapeConfig.xml

namely

<ivysettings>
  <settings defaultResolver="downloadGrapes"/>
  <resolvers>
    <chain name="downloadGrapes">
      <filesystem name="cachedGrapes">
        <ivy pattern="${user.home}/.groovy/grapes/[organisation]/[module]/ivy-[revision].xml"/>
        <artifact pattern="${user.home}/.groovy/grapes/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]"/>
      </filesystem>
      <ibiblio name="codehaus" root="http://repository.codehaus.org/" m2compatible="true"/>
      <ibiblio name="central" root="http://central.maven.org/maven2/" m2compatible="true"/>
      <ibiblio name="java.net2" root="http://download.java.net/maven/2/" m2compatible="true"/>
      <ibiblio name="hyracs-releases" root="http://obelix.ics.uci.edu/nexus/content/groups/hyracks-public-releases/" m2compatible="true"/>
    </chain>
  </resolvers>
</ivysettings>

I was able to issue the command 

bin/gremlin-server.sh -i com.thinkaurelius.titan titan-all 1.0.0

I was able to install the titan-all dependencies without error.

Now when I start the gremlin-server with the following command I was getting following error.

bin/gremlin-server.sh conf/gremlin-server.yaml

[INFO] GremlinServer -

         \,,,/

         (o o)

-----oOOo-(3)-oOOo-----

[INFO] GremlinServer - Configuring Gremlin Server from conf/gremlin-server.yaml

[INFO] MetricManager - Configured Metrics ConsoleReporter configured with report interval=180000ms

[INFO] MetricManager - Configured Metrics CsvReporter configured with report interval=180000ms to fileName=/tmp/gremlin-server-metrics.csv

[INFO] MetricManager - Configured Metrics JmxReporter configured with domain= and agentId=

[INFO] MetricManager - Configured Metrics Slf4jReporter configured with interval=180000ms and loggerName=org.apache.tinkerpop.gremlin.server.Settings$Slf4jReporterMetrics

[WARN] GraphManager - Graph [graph] configured at [conf/titan-hbase-es.properties] could not be instantiated and will not be available in Gremlin Server.  GraphFactory message: GraphFactory could not instantiate this Graph implementation [class com.thinkaurelius.titan.core.TitanFactory]

java.lang.RuntimeException: GraphFactory could not instantiate this Graph implementation [class com.thinkaurelius.titan.core.TitanFactory]

        at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:82)

        at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:70)

        at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:104)

        at org.apache.tinkerpop.gremlin.server.GraphManager.lambda$new$27(GraphManager.java:50)

        at java.util.LinkedHashMap$LinkedEntrySet.forEach(LinkedHashMap.java:663)

        at org.apache.tinkerpop.gremlin.server.GraphManager.<init>(GraphManager.java:48)

        at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:94)

        at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:88)

        at org.apache.tinkerpop.gremlin.server.GremlinServer.main(GremlinServer.java:290)

Caused by: java.lang.reflect.InvocationTargetException

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:497)

        at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:78)

        ... 8 more

Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseIOException

        at java.lang.ClassLoader.defineClass1(Native Method)

        at java.lang.ClassLoader.defineClass(ClassLoader.java:760)

        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

        at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)

        at java.net.URLClassLoader.access$100(URLClassLoader.java:73)

        at java.net.URLClassLoader$1.run(URLClassLoader.java:368)

        at java.net.URLClassLoader$1.run(URLClassLoader.java:362)

        at java.security.AccessController.doPrivileged(Native Method)

        at java.net.URLClassLoader.findClass(URLClassLoader.java:361)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

        at java.lang.ClassLoader.defineClass1(Native Method)

        at java.lang.ClassLoader.defineClass(ClassLoader.java:760)

        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

        at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)

        at java.net.URLClassLoader.access$100(URLClassLoader.java:73)

        at java.net.URLClassLoader$1.run(URLClassLoader.java:368)

        at java.net.URLClassLoader$1.run(URLClassLoader.java:362)

        at java.security.AccessController.doPrivileged(Native Method)

        at java.net.URLClassLoader.findClass(URLClassLoader.java:361)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

        at java.lang.Class.forName0(Native Method)

        at java.lang.Class.forName(Class.java:264)

        at com.thinkaurelius.titan.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:42)

        at com.thinkaurelius.titan.diskstorage.Backend.getImplementationClass(Backend.java:473)

        at com.thinkaurelius.titan.diskstorage.Backend.getStorageManager(Backend.java:407)

        at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.<init>(GraphDatabaseConfiguration.java:1320)

        at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:94)

        at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:74)

        ... 13 more

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.HBaseIOException

        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

        ... 45 more

First I thought it might be JDK issue that Gremlin required Java 8 (1.8) so I checked the current version and made sure that both JAVA_HOME as well as java -version come up with 1.8 using 

sudo update-alternatives --config java 

This still did not resolve the issue, on further probing I found that its the hbase class that was missing which was found in the  hbase-common-0.98.2-hadoop1.jar. 

Though this was found in the Titan distribution this was not pulled using the 

bin/gremlin-server.sh -i com.thinkaurelius.titan titan-all 1.0.0

I add this both to the ext/titan-all lib as well as the plugin folders. I wasn't sure why jar counts were different in all the folders titan/bin, gremlin server lib and plugin folders.

Now things started working fine.

Thanks again.

P.S I have attached the gremlin config used for titan-hbase-elasticsearch.

[Stephen Mallette]

There might be weirdness with using :install or gremlin-server.sh -i when it comes to hbase (something to do with the packaging approach of Titan itself).  :install and -i are meant as a convenience and ultimately developers my end up sorting out jar conflicts and other oddities.  there's a ticket or two on the TinkerPop side to help make the process less error prone.

--
You received this message because you are subscribed to the Google Groups "Aurelius" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aureliusgraph...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/aureliusgraphs/05e5c7bd-4b30-4628-97f3-eba12d4a914f%40googlegroups.com.