Spark OLAP Traversing is able to find the table I specified in properties.

[prasad.d...@gmail.com]

Hi,

I am new to JanusGraph and trying to do OLAP traversal on a Graph.

I dont understand why gremlin could not find the table (created with JanusGraphFactory) even when I specified 

table with property storage.hbase.table and still looking for default graph janusgraph.

My properties:

==========

cat /home/hadoop/spark-janus-hbase-axe-indv.properties

gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph

gremlin.hadoop.graphInputFormat=org.janusgraph.hadoop.formats.hbase.HBaseInputFormat

storage.backend=hbase

cache.db-cache = true

cache.db-cache-clean-wait = 20

cache.db-cache-time = 180000

cache.db-cache-size = 0.5

storage.hbase.table=spark-axe-janusgraph

storage.hostname= 172.31.40.228

spark.master=local[*]

spark.executor.memory=1g

spark.driver.cores=1

spark.submit.deployMode=client

spark.executor.instances=4

Gremlin Console

================

gremlin> hadoopGraph =GraphFactory.open('/home/hadoop/spark-janus-hbase-axe-indv.properties')

==>hadoopgraph[hbaseinputformat->no-writer]

gremlin> hadoopGraph.traversal().withComputer(SparkGraphComputer).V()

23:21:29 WARN  org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputer  - No gremlin.hadoop.graphWriter property provided and thus, persistence options are unknown -- assuming all options are possible

org.apache.hadoop.hbase.TableNotFoundException: Table 'janusgraph' was not found, got: hypergraph.

Type ':help' or ':h' for help.

Display stack trace? [yN]

Any help is much appreciated. Thanks in advance....

Regards,

Prasad Dokuparthi.

[HadoopMarc]

Hi Prasad,

Nothing seems wrong, so maybe check:

config = new PropertiesConfiguration('/home/hadoop/spark-janus-hbase-axe-indv.properties')
config.getProperty('storage.hbase.table')

Cheers,     Marc

Op dinsdag 16 januari 2018 00:57:08 UTC+1 schreef prasad.d...@gmail.com:

[prasad.d...@gmail.com]

Thanks a lot for your reply.

The configuration I am giving seems to be reflecting but not working.

gremlin> graph = GraphFactory.open('/home/hadoop/spark-janus-hbase-axe-indv.properties')

==>hadoopgraph[hbaseinputformat->no-writer]

gremlin> 

gremlin> graph.configuration()

==>[spark.executor.memory,1g]

==>[spark.master,local[*]]

==>[spark.driver.cores,1]

==>[gremlin.graph,org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph]

==>[cache.db-cache-size,0.5]

==>[spark.submit.deployMode,client]

==>[cache.db-cache-clean-wait,20]

==>[cache.db-cache,true]

==>[storage.backend,hbase]

==>[storage.hostname,172.31.47.151]

==>[gremlin.hadoop.graphReader,org.janusgraph.hadoop.formats.hbase.HBaseInputFormat]

==>[storage.hbase.table,spark-axe-janusgraph]

==>[gremlin.hadoop.inputLocation,/user/hbase/data/default/spark-axe-janusgraph]

==>[spark.executor.instances,4]

==>[cache.db-cache-time,180000]

gremlin> 

gremlin> 

Regards,

Prasad.

[HadoopMarc]

OK, maybe try to add the following property (also present in the Cassandra example in conf/hadoop-graph):

gremlin.hadoop.inputLocation=none

HTH,   Marc

Op dinsdag 16 januari 2018 17:43:35 UTC+1 schreef prasad.d...@gmail.com:

[prasad.d...@gmail.com]

Hi Marc,

Thanks a lot, I got it worked with 

janusgraphmr.ioformat.conf.storage.hbase.table=spark-acxiom-janusgraph

But I am surprised to see that JanusGraph is not picking with storage.hbase.table 

and there is no good documentation in one place to explain how configuration should be set up to make gremlin work with spark.

There is tons of documentation present to traverse, process and do operations on the stupid default JanusGraph with 6 vertices.

but what to do with that if every user is facing trouble on how to setup gremlin configuration , create a userdefined graph and open it and traverse it. 

I cant even find a sample code to load data to spark with InputRDD menthod anywhere.

Sorry I dont want to be rude, but series of exceptions and not getting help is causing frustration. 

Thanks a lot for your help.

Regards,

Prasad.

[HadoopMarc]

Hi Prasad,

Your comments are fair enough. The thing is, many developers familiar with the old Titan-0.5.x, have had to wait very long for the JanusGraph releases and are eager to update their own projects. Nevertheless, everyone should realize that the JanusGraph community needs new users to keep it healthy on the long term. I am busy right now to bulkload the Friendster graph (https://snap.stanford.edu/data/com-Friendster.html) into JanusGraph and hope to blog about it in the near future and provide users with a reference big data graph example.

The janusgraphmr.ioformat.conf prefix for hadoop is tricky indeed and I feel bad that I missed that one.

Cheers,    Marc

Op dinsdag 16 januari 2018 22:01:01 UTC+1 schreef prasad.d...@gmail.com: