I have been attempting to reproduce the steps in the documentation for SASL_PLAINTEXT described here:
http://kafka.apache.org/documentation/#security_sasl_plainSo far I have failed to get it to work despite all my attempts to follow the documentation to the letter.
ZK and broker starts OK. I create a test topic, launch a producer and as soon as I start typing into the terminal I get this output
[2017-09-22 11:17:08,948] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-09-22 11:17:09,002] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-09-22 11:17:09,058] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-09-22 11:17:09,113] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-09-22 11:17:09,169] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-09-22 11:17:09,226] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-09-22 11:17:09,332] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
Looking up the source code for the corresponding log printout I found the following comment:
// 'processDisconnection' generates warnings for misconfigured bootstrap server configuration
// resulting in 'Connection Refused' and misconfigured security resulting in authentication failures.
// The warning below handles the case where connection to a broker was established, but was disconnected
// before metadata could be obtained.
So it appears I got the configuration wrong somewhere however I just can't find where.
Here are exact steps I went through:1. Created /usr/local/etc/kafka/server-jaas.conf file with the following content:
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="kafka"
password="kafka"
user_kafka="kafka";
};
2. Updated KAFKA_OPTS to include the file into JVM options:
export KAFKA_OPTS="-Djava.security.auth.login.config=/usr/local/etc/kafka/client-jaas.conf"
3. Made the following updates to the /usr/local/etc/kafka/server.properties:
listeners=SASL_PLAINTEXT://localhost:9092
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
4. Made the following updates to /usr/local/etc/kafka/produce.properties and /usr/local/etc/kafka/consumer.properties:
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="kafka" \
password="kafka";
That's all. After seeing the failure I tried to configure producer/consumer via a JAAS file in a similar way I configured the broker however that made no difference, I still see the endless printout indicating I am being disconnected. Any thoughts/ideas?
P.S. For the tests I used the command line producer/consumer script provided with Kafka installation and the steps from getting started section of the docs:
http://kafka.apache.org/documentation/#quickstart_startserver1. Start ZK:
zookeeper-server-start config/zookeeper.properties
2. Start broker
kafka-server-start config/server.properties
3. Create test topic:
kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
4. Publish:
kafka-console-producer --broker-list localhost:9092 --topic test