About configuring Kafka 2.10-0.10.2 using SASL

[Sharat Joshi]

Hi Kafka Users,

I am trying to setup a Simple Authentication Mechanism for my Kafka Instance running on my Virtual Box VM.

I am facing a lot of difficulty in starting the Zookeeper 3.4.10.

The scenario is like this...

I have a single admin User called sharjosh who's starting both the Zookeeper and Kafka processes on the VM.

I tried to use the following command to start the zookeeper

~/zookeeper-3.4.10$ export JVMFLAGS='-Djava.security.auth.login.config=/home/sharjosh/kafka_2.10-0.10.2.1/config/zk_server_jaas.conf'; bin/zkServer.sh start-foreground conf/zoo.cfg

and get

 at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53) at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:87) at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:117) at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:82) at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:200)java.io.IOException: No JAAS configuration section named 'Server' was foundin '/home/sharjosh/kafka_2.10-0.10.2.1/config/zk_server_jaas.conf'.2017-06-04 22:43:31,324 [myid:] - ERROR [main:ZooKeeperServerMain@64] - Unexpected exception, exiting abnormally2017-06-04 22:43:31,323 [myid:] - ERROR [main:ServerCnxnFactory@199] - No JAAS configuration section named 'Server' was foundin '/home/sharjosh/kafka_2.10-0.10.2.1/config/zk_server_jaas.conf'.

My JAAS Config file for Zookeeper is

Server {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    useTicketCache=true
    keyTab="/home/sharjosh/kafkaSecurity/sharjosh.keytab"
    principal="sharjosh@localhost"
};

My zoo.cfg file is

# The number of milliseconds of each tick
tickTime=2000

# The number of ticks that the initial
# synchronization phase can take
initLimit=10

# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5

# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/home/sharjosh/zookeeper-3.4.10/data

# the port at which the clients will connect
clientPort=2181

# the maximum number of client connections.
# increase this if you need to handle more clients
maxClientCnxns=500

# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
autopurge.snapRetainCount=3

# Purge task interval in hours
# Set to "0" to disable auto purge feature
autopurge.purgeInterval=1

# https://cwiki.apache.org/confluence/display/ZOOKEEPER/Client-Server+mutual+authentication
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000

My Kafka startup command is

export KAFKA_OPTS='-Djava.security.krb5.conf=/etc/krb5.conf -Djava.security.auth.login.config=/home/sharjosh/kafka_2.10-0.10.2.1/config/kafka_server_jaas.conf -Dzookeeper.sasl.client=zKClient'; ./kafka-server-start.sh /home/sharjosh/kafka-0.10.2.1-src/config/server.properties

My Kafka server's JAAS file is

KafkaServer {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    serviceName=kafka
    useTicketCache=true
    keyTab="/home/sharjosh/kafkaSecurity/sharjosh.keytab"
    principal="sharjosh@localhost"
};

zKClient {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    serviceName=zookeeper
    useTicketCache=true
    keyTab="/home/sharjosh/kafkaSecurity/sharjosh.keytab"
    principal="sharjosh@localhost"
};

My security settings in kafka's server.properties

listeners=SASL_PLAINTEXT://:9092 

advertised.listeners=SASL_PLAINTEXT://:9092

security.inter.broker.protocol=SASL_PLAINTEXT
sasl.enabled.mechanisms=GSSAPI,PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN

zookeeper.connect=localhost:2181

# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=6000

On the other hand in Kafka Broker startup also I get errors as there's no Zookeeper available.

org.apache.kafka.common.KafkaException: Exception while loading Zookeeper JAAS login context 'Client'
 at org.apache.kafka.common.security.JaasUtils.isZkSecurityEnabled(JaasUtils.java:154)
 at kafka.server.KafkaServer.initZk(KafkaServer.scala:310)
 at kafka.server.KafkaServer.startup(KafkaServer.scala:187)
 at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39)
 at kafka.Kafka$.main(Kafka.scala:67)
 at kafka.Kafka.main(Kafka.scala)
Caused by: java.lang.SecurityException: java.io.IOException: Configuration Error:
 Line 9: expected [option key]
 at sun.security.provider.ConfigFile$Spi.<init>(ConfigFile.java:137)
 at sun.security.provider.ConfigFile.<init>(ConfigFile.java:102)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
 at java.lang.Class.newInstance(Class.java:442)
 at javax.security.auth.login.Configuration$2.run(Configuration.java:255)
 at javax.security.auth.login.Configuration$2.run(Configuration.java:247)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.login.Configuration.getConfiguration(Configuration.java:246)
 at org.apache.kafka.common.security.JaasUtils.isZkSecurityEnabled(JaasUtils.java:151)
 ... 5 more
Caused by: java.io.IOException: Configuration Error:
 Line 9: expected [option key]

...

So please advice me on what I am missing in the Configs to properly start Zookeeper to move forward. I have put in many days of investigation leading to pretty much the same result.

I am eagerly hoping the group and its users might have already conquered it and would help me. Waiting to hear back from you.

Thanks a lot,

Sharat

[Ryan Pridgeon]

You are missing a semi-colon in your jaas conf. The last property in each jaas configuration section needs a semi-colon in addition to the one at after the closing curly-bracket. 

i.e. 

Server {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    useTicketCache=true
    keyTab="/home/sharjosh/kafkaSecurity/sharjosh.keytab"
    principal="sharjosh@localhost";

};

Add the missing semi-colon to all of your jaas conf files and try agian. 

I only see the Krb5LoginModule configured in your jaas files. You should set sasl.enabled.mechanisms=GSSAPI, otherwise you are likely to hit an exception when the PlainLoginModule config can not be found. 

PLAINTEXT is different from PLAIN.

Lastly since there are no PlainLoginModules configured your inter-broker sasl mechanism should be gssapi as well. Either that or you should configure PlainLoginModules in you jaas conf files as well. 

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/f8765a65-b2c2-4b56-8687-79585afbec97%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.