SparkseeHA setup

[Laura Daian]

Hello

I have tried to use SparkseeHA in order to run sparksee in a cluster. I have installed the Apache Zookeeper and HAProxy and tried to configure them as in the tutorial offered.
My SPraksee application has the following configuration:

sparksee.license=****
sparksee.log.file=HelloSparksee.log
parksee.io.cache.maxsize=2048
sparksee.ha=true
sparksee.ha.ip=***.**.**:7777
sparksee.ha.coordinators=***.**.**.205:2181,***.**.**.206:2181,***.**.**.207:2181
sparksee.ha.sync=600s
sparksee.ha.master.history=24H

When I try to run my Sparksee application I get a runtime error that doesn't really tell me what the problem is:

java.lang.RuntimeException: C++ sparksee::gdb::Error exception thrown

The application works well in a non-cluster environment. As soon as I add the last 5 lines in the Sparksee configuration file I receive the error.
I have Sparksee version v5.1.0 and Zookeeper v3.4.6. In the documentation it is only mentioned about Sparksee v4.7 and ZooKeeper v3.4.3. Are my chosen versions compatible or did I make some configuration errors?


Thanks for your help,
Laura

[c3po.ac]

Hi,

When a sparksee Error exception is generated, it should be the equivalent error message in the log.
Please, check your logfile for "SEVERE" error messages.

It's probably not a versions problem.

Best regards.


El dilluns 27 d’octubre de 2014 12:10:19 UTC+1, Laura Daian va escriure:

[Laura Daian]

Hey

I looked at the logfile mentioned in the configuration file. There is no SEVRE erro. The only thing I have there is :

2014-10-27 15:02:08.197|SPARKSEE[INFO] ** START ** 2014-10-27 15:02:08.197 Software version: 5.1.0
2014-10-27 15:02:08.197|SPARKSEE[INFO] Decoded license: Software version [5] ...
2014-10-27 15:02:08.432|GraphPool[INFO] Closed db/bsbm.gdb
2014-10-27 15:02:08.433|GraphPool[WARNING] Removed db/bsbm.gdb.tmp

Laura

[c3po.ac]

Hi,

You could catch the exception to see it's message. But the most probable cause of the error is that the Sparksee Instance is unable to connect to the Zookeeper Coordinators.

Please check if your zookeeper cluster is correctly configured and running.
You could use the zookeeper scripts to check it:
https://zookeeper.apache.org/doc/r3.1.2/zookeeperStarted.html#sc_ConnectingToZooKeeper

Best regards.


El dilluns 27 d’octubre de 2014 15:12:46 UTC+1, Laura Daian va escriure:

[Laura Daian]

Hi,

Thanks for the answer. The zookeeper wasn't configured correctly. I didn't create the "myid" file for my servers. Now they start and seems to be working fine.

I have my application on one of the servers. I'm trying to import some data on the cluster environment now. I took the exact code what I used for one node (create the database and a new session, get the graph and add nodes and edges to the graph). And the program just seems to hang. It stoppes running at the line where the database is created:

Database db = sparksee.create("db/bsbm.gdb", "bsbm");

It creates the bsbm.gdb file plus a log file and a tmp file just on the running node and nothing on the other nodes. What am I doing wrong? why doesn't it succeed to pass the creation of the database?

Best regards
Laura

[c3po.ac]

Hi,

There may be a problem with the master configured at zookeeper, you could send us the Zookeper directory contents using this tool:
https://zookeeper.apache.org/doc/r3.1.2/zookeeperStarted.html#sc_ConnectingToZooKeeper

To see what could be wrong in the configuration.

At your convenience send it to dam...@sparsity-technologies.com or post it here.

Best regards.


El divendres 31 d’octubre de 2014 13:31:28 UTC+1, Laura Daian va escriure:

[sparkseegdb]

In case it may be useful for other Sparksee users dealing with the HA functionality, on our follow up by email we suggested the following:

- Create and load your dataset on the master alone.

- Then open the slaves which will expect that the DB is already created, the replicas would be created accordingly.

Hope this is helpful!

Sparksee team

El viernes, 31 de octubre de 2014 16:06:32 UTC+1, c3po.ac escribió:

Hi,

There may be a problem with the master configured at zookeeper, you could send us the Zookeper directory contents using this tool:
https://zookeeper.apache.org/doc/r3.1.2/zookeeperStarted.html#sc_ConnectingToZooKeeper

To see what could be wrong in the configuration.

At your convenience send it to damaris@sparsity-technologies.com or post it here.