Solr cloud with multiple ZK instances

49 views
Skip to first unread message

Apoorv Naik

unread,
Sep 27, 2018, 11:38:06 PM9/27/18
to JanusGraph users
Looking at the documentation around using Solr in cloud mode, looks like there's only one zookeeper URL that's supported. How can I configure the JanusGraph instance to use multiple zookeeper instances instead of just one ?

Jerry He

unread,
Sep 28, 2018, 12:02:41 AM9/28/18
to JanusGraph users list
Hi, Apoorv

Can you explain what you want?  Do you mean multiple servers for a Zookeeper ensemble by 'multiple zookeeper instances'?  Or you mean multiple ensembles?

Thanks,

Jerry

On Thu, Sep 27, 2018 at 8:38 PM, Apoorv Naik <naik....@gmail.com> wrote:
Looking at the documentation around using Solr in cloud mode, looks like there's only one zookeeper URL that's supported. How can I configure the JanusGraph instance to use multiple zookeeper instances instead of just one ?

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To post to this group, send email to janusgraph-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/2c870817-b0f0-413e-955b-7739689eb0db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Apoorv Naik

unread,
Sep 28, 2018, 12:14:32 AM9/28/18
to JanusGraph users
Hi Jerry,

I've a cluster where I've three Zookeepers deployed ( single ensemble) , the moment the first one goes down the Janusgraph instance becomes unusable and goes into an exception loop. So what I was trying to achieve was a failover to the other zookepeer server in the ensemble.

On Thursday, September 27, 2018 at 9:02:41 PM UTC-7, Jerry He wrote:
Hi, Apoorv

Can you explain what you want?  Do you mean multiple servers for a Zookeeper ensemble by 'multiple zookeeper instances'?  Or you mean multiple ensembles?

Thanks,

Jerry
On Thu, Sep 27, 2018 at 8:38 PM, Apoorv Naik <naik....@gmail.com> wrote:
Looking at the documentation around using Solr in cloud mode, looks like there's only one zookeeper URL that's supported. How can I configure the JanusGraph instance to use multiple zookeeper instances instead of just one ?

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To post to this group, send email to janusgra...@googlegroups.com.

Jerry He

unread,
Sep 28, 2018, 12:58:21 AM9/28/18
to JanusGraph users list
Hi, Apoorv

Try:

index.search.solr.zookeeper-url=aid1:2181/solr,aid2:2181/solr,aid3:2181/solr

Replace hostnames and other info, with the members in you assemble.

Thanks,

Jerry 

To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To post to this group, send email to janusgraph-users@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/6b696132-af2a-436e-93c4-49e41a16289c%40googlegroups.com.

Apoorv Naik

unread,
Sep 28, 2018, 1:04:00 AM9/28/18
to janusgra...@googlegroups.com
Yes this is exactly what we're doing but if the first ZK server in the ensemble goes down the server runs into an endless retry loop, doesn't failover to other hosts at all.

Apoorv Naik


To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/6b696132-af2a-436e-93c4-49e41a16289c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To post to this group, send email to janusgra...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/CAHkeaHVnHgzjTxsk7Y87GWFYRqCUh%3Dm_F7TOB4jvJQ79%2BHYKWw%40mail.gmail.com.

Jerry He

unread,
Sep 28, 2018, 3:03:24 PM9/28/18
to JanusGraph users list
I think JanusGraph simply passes the  zookeeper-url string to the Solr cloud client without doing any parsing, and the string format is what Solr needs.
It is strange and puzzling that Solr's use of zookeeper client is not doing the re-connect.  Zookeeper client itself would do the fail-over reconnect (assume the correct servers and parameters are given to it).

Thanks,

Jerry

To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To post to this group, send email to janusgraph-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To post to this group, send email to janusgraph-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/CAHkeaHVnHgzjTxsk7Y87GWFYRqCUh%3Dm_F7TOB4jvJQ79%2BHYKWw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To post to this group, send email to janusgraph-users@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/CAOT1%3D38Sbe9q7x1jmMeV6kAGHH-9vHxFyZ8tp6Jg01ANCea3AA%40mail.gmail.com.

Sarath Subramanian

unread,
Oct 2, 2018, 5:19:21 PM10/2/18
to JanusGraph users
Jerry,


final String zookeeperUrl = config.get(SolrIndex.ZOOKEEPER_URL);

Always fetches single  string value, even if comma-separated zookeeper host values are specified. 

Maybe the fix is changing the data type from String to String array below?

public static final ConfigOption<String> ZOOKEEPER_URL = new ConfigOption<>(SOLR_NS,"zookeeper-url", "URL of the Zookeeper instance coordinating the SolrCloud cluster", ConfigOption.Type.MASKABLE, "localhost:2181");


Thanks,
Sarath 

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To post to this group, send email to janusgra...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
To post to this group, send email to janusgra...@googlegroups.com.

madhan.n...@gmail.com

unread,
Oct 2, 2018, 5:43:32 PM10/2/18
to JanusGraph users
The issue is trigged by CommonsConfigurations implementation, which calls getString(key) for String type values. When the value for the key is an array, getString(key) returns the first element - which causes JanusGraph to use only the first zookeeper host:port.

Changing SolrIndex.ZOOKEEPER_URL from "String" to "String[]" will end up calling getStringArray() , which will return all entries of the array.

Jerry He

unread,
Oct 3, 2018, 1:42:58 AM10/3/18
to JanusGraph users list, JanusGraph developer list
Hi, Sarath and Madhan

Your analysis makes sense.

I did a quick test.
From gremlin shell: 
gremlin> graph.getConfiguration().getLocalConfiguration().getProperty('index.search.solr.zookeeper-url')
It only returns the first server out of the three servers in  'index.search.solr.zookeeper-url' from the property file.
On the other hand,
gremlin> graph.getConfiguration().getLocalConfiguration().getProperty('storage.hostname')
returns the list of three zookeeper servers for the hbase property.  storage.hostname is a String[] config option.  

We need to fix it, as you suggested.

Thanks for digging into it.

Jerry 


To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To post to this group, send email to janusgraph-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/125d6193-9e40-41bf-a569-085c7848f479%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages