How to setup exhibitor for ZK ensemble of 3 nodes running on one machine

[pooja s]

Hi

Iam new to using zookeeper and exhibitor. I set up  a 3 node ZK cluster on my VM. I wanted to run exhibitor service for the cluster ( preferably war) on the same node . Is it possible to do this.

I tried 3 instances of tomcat with war deployed  however the dataDir and logdir all start pointing to one location.

 

Conf 

tickTime=2000

initLimit=10

syncLimit=5

dataDir=~/zookeeper1/zookeeper1/data

dataLogDir=~/zookeeper1/zookeeper1/txlog

clientPort=2181

snapCount=1000000

autopurge.snapRetainCount=3

autopurge.purgeInterval=1

maxClientCnxns=30     

server.1=localhost:2888:3888

server.2=localhost:2889:3889

server.3=localhost:2890:3890

Conf 2

tickTime=2000

initLimit=10

syncLimit=5

dataDir=~/zookeeper2/zookeeper2/data

dataLogDir=~/zookeeper2/zookeeper2/txlog

clientPort=2182

snapCount=1000000                                

autopurge.snapRetainCount=3                                  

autopurge.purgeInterval=1                                          

maxClientCnxns=30

server.1=localhost:2888:3888

server.2=localhost:2889:3889

server.3=localhost:2890:3890

conf 3 

tickTime=2000

initLimit=10

syncLimit=5

dataDir=~/zookeeper3/zookeeper3/data

dataLogDir=~/zookeeper3/zookeeper3/txlog

clientPort=2183

snapCount=1000000

autopurge.snapRetainCount=3

autopurge.purgeInterval=1

maxClientCnxns=30     

server.1=localhost:2888:3888

server.2=localhost:2889:3889

server.3=localhost:2890:3890

[Jordan Zimmerman]

Currently, Exhbitor does not handle multiple ZK instances on the same machine. Exhibitor is designed to replicate ZK configuration for every ZK instances.

-JZ

---

From: pooja s pooja s
Reply: exhibit...@googlegroups.com exhibit...@googlegroups.com
Date: January 20, 2014 at 3:07:38 AM
To: exhibit...@googlegroups.com exhibit...@googlegroups.com
Subject:  How to setup exhibitor for ZK ensemble of 3 nodes running on one machine

--
You received this message because you are subscribed to the Google Groups "exhibitor-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to exhibitor-use...@googlegroups.com.
To post to this group, send email to exhibit...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/exhibitor-users/c975168e-f78b-4e35-999a-5813183df20d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[pooja s]

Thanks for the quick response.

Couple of questions

1. Before we build the war the exhibitor property file needs to be modified However in my case i need to have different values for different environment is there a way i can do this  ?
   
2. Can you let me know the properties to be set in exhibitor.properties file   for  "All At Once" in Automatic Instance Management Configuration and for automatic instant restart in control panel
3. Is it possible that exhibitor just takes the zookeeper config ( zoo.cfg,log4j.prop,java.env) based on installDir information and uses that information to start stop the service and not rewrite the zoo.cfg 

Thanks 

Pooja

[Jordan Zimmerman]

1. Before we build the war the exhibitor property file needs to be modified However in my case i need to have different values for different environment is there a way i can do this  ?

Is there a reason you’re using a WAR? Otherwise, that’s how it currently works.

1. Can you let me know the properties to be set in exhibitor.properties file   for  "All At Once" in Automatic Instance Management Configuration and for automatic instant restart in control panel

AUTO_MANAGE_INSTANCES: com.netflix.exhibitor.auto-manage-instances

AUTO_MANAGE_INSTANCES_APPLY_ALL_AT_ONCE: com.netflix.exhibitor.auto-manage-instances-apply-all-at-once

1. Is it possible that exhibitor just takes the zookeeper config ( zoo.cfg,log4j.prop,java.env) based on installDir information and uses that information to start stop the service and not rewrite the zoo.cfg 

No - not currently.

---

From: pooja s pooja s
Reply: exhibit...@googlegroups.com exhibit...@googlegroups.com

Date: January 20, 2014 at 6:14:17 PM
To: exhibit...@googlegroups.com exhibit...@googlegroups.com
Subject:  Re: How to setup exhibitor for ZK ensemble of 3 nodes running on one machine

--
You received this message because you are subscribed to the Google Groups "exhibitor-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to exhibitor-use...@googlegroups.com.
To post to this group, send email to exhibit...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/exhibitor-users/9060e580-3bb4-491b-8c34-5193b576b0f2%40googlegroups.com.

[pooja s]

Thanks for the response.

We are using war so that we can piggyback on existing jboss deployment in production. We want  that when exhibitor starts it has necessary zoo.cfg.java.env,log4j.properties to start the service ( no manual intervention)  and it has automatic instance restart  set to manage the ZK instance.

With war i was facing couple of issue . 

1. If we do not rename the generated war to ROOT.war  it does not work.
2. I am attaching the exhibitor.properties for which i generated the war however i get the following  error ( log attached along-with). Is there something wrong with the configuration ?

Exhibitor .properties

exhibitor-configtype=file 

exhibitor-port=8888 

exhibitor-configcheckms=30000

exhibitor-loglines=1000

exhibitor-nodemodification=true

exhibitor-servo=true

exhibitor-timeout=30000

exhibitor-cleanup-period-ms=72000000

#exhibitor-backup-period-ms=72000000

#exhibitor-auto-manage-instances=true

#exhibitor-backup-max-store-ms=72000000

exhibitor-cleanup-max-files=3

#exhibitor-filesystembackup=true

exhibitor-check-ms=30000 

exhibitor-client-port=2181

exhibitor-connect-port=2888

exhibitor-election-port=3888

exhibitor-fsconfigdir=~/dev/

#exhibitor-log-index-directory~/Downloads/zookeeper5/index/ 

exhibitor-zookeeper-install-directory=~/Downloads/zookeeper5/

exhibitor-zookeeper-data-directory=~/zookeeper5/zookeeper5/data/ 

exhibitor-zookeeper-log-directory=~/zookeeper5/zookeeper5/txlog/ 

#exhibitor-backup-extra="~zookeeper5/zookeeper5/backup/" 

#exhibitor-zoo-cfg-extra=

#exhibitor-java-environment=

#exhibitor-log4j-properties=

3  Can you provide some sample for setting in the properties file   java.env and log4j (exhibitor-java-environment,exhibitor-log4j-properties)

Is this how i should specify exhibitor-zoo-cfg-extra 

ex  

exhibitor-zoo-cfg-extra-tickTime=2000  

exhibitor-zoo-cfg-extra-initLimit=10 

exhibitor-zoo-cfg-extra-syncLimit=5 

exhibitor-zoo-cfg-extra-snapCount=1000000  

exhibitor-zoo-cfg-extra-autopurge.snapRetainCount=3 

exhibitor-zoo-cfg-extra-autopurge.purgeInterval=1 

exhibitor-zoo-cfg-extra-maxClientCnxns=30

Regards

Pooja

[pooja s]

Below is config i used to get exhibitor working with WAR and setting exhibitor.properties external to war generation process.

• Generate the exhibitor war from project
  
• Deploy the war to jboss server
• Set exhibitor.properties when running the JBOSS container.
   JAVA_OPTS="$JAVA_OPTS -Dexhibitor-fsconfigdir=/nas/ -Dexhibitor-defaultconfig=/opt/zookeper/startup.properties -Dexhibitor-port=8080 -Dexhibitor-configcheckms=30000 -Dexhibitor-loglines=10000  -Dexhibitor-timeout=30000 -Dexhibitor-filesystembackup=true  -Dexhibitor-nodemodification=true -Dexhibitor-servo=true -Dexhibitor-hostname=host "
  • Create startup.property file at $ZK_HOME/startup.properties
    Default values for Zookeeper/exhibitor config (startup.properties  content)
    cleanup-period-ms=72000000
    backup-period-ms=36000000
    auto-manage-instances=true
    backup-max-store-ms=72000000
    cleanup-max-files=3
    check-ms=30000
    client-port=2181
    connect-port=2888
    election-port=3888
    log-index-directory=/opt/zookeper/index/
    zookeeper-install-directory=/opt/zookeper/
    zookeeper-data-directory=/opt/zookeper/data/
    zookeeper-log-directory=/opt/zookeper/txlog/
    servers-spec=0\:10.249.133.39,1\:10.249.133.40,2\:10.249.133.41
    backup-extra=/opt/zookeper/backup/
    zoo-cfg-extra=tickTime\=2000&initLimit\=10&syncLimit\=5&snapCount\=1000000&autopurge.snapRetainCount\=3&autopurge.purgeInterval\=1&maxClientCnxns\=30
    
    
• Startup JBOSS service  and hit URL : http://host1:8080/exhibitor/v1/ui/index.html to view ZK ensemble