Manual Install and Configure Confluent 5.0 platform questions

[Xu, Qian]

Hi Team,

I am new to Confluent platform.

I am Manually  installing the Confluent 5.0.0 Platform  using   confluent-oss-5.0.0-2.11.zip Archive.  I followed the instruction from URL:

https://docs.confluent.io/5.0.0/installation/installing_cp/zip-tar.html

here are some configuration questions:

A: I have decompress the zip file,   and have the directories  /bin/etc/lib/share/src/,

but the /logs/ folder  is not showing up.  should I create a /logs/ folder manually or is there any Authorization issue ?

B: ZooKeeper

Here is my modified /etc/kafka/zookeeper.properties file:

tickTime=2000

dataDir=/var/lib/zookeeper/

clientPort=2181

initLimit=5

syncLimit=2

server.1=localhost:2888:3888

server.2=localhost:2889:3889

server.3=localhost:2890:3890

autopurge.snapRetainCount=3

autopurge.purgeInterval=24

and created myid file in  ZooKeeper log directory  /var/lib/zookeeper/

------------------------------------------------------------------------------------

C: Kafka

1.Customize the /etc/kafka/server.properties  file:

zookeeper.connect=localhost:2181

#broker.id=0

broker.id.generation.enable=true

2,: Create the  /etc/kafka/server1.properties  file:

broker.id=1

broker.id.generation.enable=true

listeners=https://localhost:9092

advertised.listeners=https://localhost:9092

3. Create the  /etc/kafka/server2.properties  file:

broker.id=2

broker.id.generation.enable=true

listeners=https://localhost:9093

advertised.listeners=https://localhost:9093

4. Create the  /etc/kafka/server3.properties  file:

broker.id=3

broker.id.generation.enable=true

listeners=https://localhost:9094

advertised.listeners=https://localhost:9094

D:  Control Center:

1. Customize the etc/confluent-control-center/control-center.properties:

bootstrap.servers=localhost:9092,localhost:9093,localhost:9094

confluent.controlcenter.data.dir=/tmp/confluent/control-center

zookeeper.connect=localhost:9092,localhost:9093,localhost:9094

2.  customize the  /etc/kafka/server.properties:

metric.reporters=io.confluent.metrics.reporter.ConfluentMetricsReporter

confluent.metrics.reporter.bootstrap.servers=localhost:9092

confluent.metrics.reporter.topic.replicas=1

after  Start ZooKeeper server, received error message:

C:\VZ_Confluent\confluent-oss-5.0.0-2.11\confluent-5.0.0>bin\windows\zookeeper-server-start.bat etc\kafka\zookeeper.properties

[2018-09-11 09:14:41,849] INFO Reading configuration from: etc\kafka\zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)

[2018-09-11 09:14:41,920] INFO Resolved hostname: localhost to address: localhost/127.0.0.1 (org.apache.zookeeper.server.quorum.QuorumPeer)

[2018-09-11 09:14:41,920] INFO Resolved hostname: localhost to address: localhost/127.0.0.1 (org.apache.zookeeper.server.quorum.QuorumPeer)

[2018-09-11 09:14:41,925] INFO Resolved hostname: localhost to address: localhost/127.0.0.1 (org.apache.zookeeper.server.quorum.QuorumPeer)

[2018-09-11 09:14:41,927] INFO Defaulting to majority quorums (org.apache.zookeeper.server.quorum.QuorumPeerConfig)

[2018-09-11 09:14:41,932] ERROR Invalid config, exiting abnormally (org.apache.zookeeper.server.quorum.QuorumPeerMain)

org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing etc\kafka\zookeeper.properties

        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:156)

        at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:104)

        at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)

Caused by: java.lang.IllegalArgumentException: \var\lib\zookeeper\myid file is missing

        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:408)

        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:152)

        ... 2 more

Invalid config, exiting abnormally

I appreciate if you can review the configuration files and provide troubleshooting?

Thanks!

Jennifer Xu

[Xu, Qian]

[Amit Sahu]

Hi, 

You're missing zookeeper id file. 

https://www.thegeekstuff.com/2016/10/zookeeper-cluster-install/

Please refer to above article.. 

Thanks

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CANzMyZ8aPC9S%3DBUmMbO76fOLMxxUmseF5h%2BomC9PL4dNDQDAag%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Xu, Qian]

Hi Amit:

Thank you very much for the direction.

Our develop environment is Windows 10,  do you have instruction for setup Apache Zookeeper Cluster on Multiple Node in Window ?

especially how to "Create Unique Zookeeper Id on Individual Nodes in window".

Thanks!

Jennifer

On Wed, Sep 12, 2018 at 3:15 PM, Amit Sahu <amitsahu...@gmail.com> wrote:

Hi, 

You're missing zookeeper id file. 

https://www.thegeekstuff.com/2016/10/zookeeper-cluster-install/

Please refer to above article.. 

Thanks

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.

To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CANzMyZ8aPC9S%3DBUmMbO76fOLMxxUmseF5h%2BomC9PL4dNDQDAag%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CAL%2BJYYFX2-W_sK%3Dk0MJ%2B-z39JrT7VnFxFDXD0vzrD4FxVMOGKg%40mail.gmail.com.

[Amit Sahu]

Hi Jennifer,

yes, we too did a small POC, by setting up a 3-node Kafka cluster windows before moving to Linux based Prod/UAT env.

But I can't share the artifacts on a public platform, But i'll be more than happy to share some insights regarding the same one on one.

My E-mail: amitsahu...@gmail.com

Regards,

Amit

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CANzMyZ8aPC9S%3DBUmMbO76fOLMxxUmseF5h%2BomC9PL4dNDQDAag%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CAL%2BJYYFX2-W_sK%3Dk0MJ%2B-z39JrT7VnFxFDXD0vzrD4FxVMOGKg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CANzMyZ8S1rqnQvFGOWQuE7BtdNzZNqdhLmdJ4Yj_NiR%3DinY1TA%40mail.gmail.com.