Authorization issues with Kafka LDAP Authorizer

[JITU KUMAR]

We have a secure kafka cluster running and now I am trying to implement LDAP group-based authorization using Active directory. 

I followed steps mentioned here: https://docs.confluent.io/current/confluent-security-plugins/docs/kafka/quickstart.html and I have currently set the below ACLs on a test topic:

Group:KAFKA_TEST has Allow permission for operations: Create from hosts: *

Group:KAFKA_TEST has Allow permission for operations: Write from hosts: *

Group:KAFKA_TEST has Allow permission for operations: Describe from hosts: *

User: XYZ1234 is part of the above AD group: KAFKA_TEST and I get the below successfully authenticated message when I try to publish a message to the topic,

"INFO Successfully authenticated client: authenticationID=XYZ...@DOMAIN.COM; authorizationID=XYZ...@DOMAIN.COM. (org.apache.kafka.common.security.authenticator.SaslServerCallbackHandler)"

But at the same time I am seeing authorization errors like these,

INFO Principal = User:XYZ1234 is Denied Operation = Describe from host = 153.13.204.100 on resource = Topic:LITERAL:testtopic (kafka.authorizer.logger)

Since it was authenticating successfully against XYZ...@DOMAIN.COM and the authorization error above only had XYZ1234, I thought of adding "sasl.kerberos.principal.to.local.rules=RULE:[1:$1@$0], DEFAULT" to the server.properties file which, according to my understanding should have mapped to XYZ...@DOMAIN.COM instead of the default XYZ1234. But, this change gave the below error,

ERROR Exception while processing request from 172.16.216.41:9094-153.13.204.100:57106-7 (kafka.network.Processor)

org.apache.kafka.common.KafkaException: Failed to set name for 'XYZ...@DOMAIN.COM' based on Kerberos authentication rules.

        at org.apache.kafka.common.security.authenticator.DefaultKafkaPrincipalBuilder.applyKerberosShortNamer(DefaultKafkaPrincipalBuilder.java:134)

        at org.apache.kafka.common.security.authenticator.DefaultKafkaPrincipalBuilder.build(DefaultKafkaPrincipalBuilder.java:120)

        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.principal(SaslServerAuthenticator.java:286)

        at org.apache.kafka.common.network.KafkaChannel.principal(KafkaChannel.java:123)

        at org.apache.kafka.common.network.KafkaChannel.newRequestContext(KafkaChannel.java:383)

        at kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:694)

        at kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:688)

        at scala.collection.Iterator$class.foreach(Iterator.scala:891)

        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)

        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)

        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)

        at kafka.network.Processor.processCompletedReceives(SocketServer.scala:688)

        at kafka.network.Processor.run(SocketServer.scala:584)

        at java.lang.Thread.run(Thread.java:748)

Caused by: org.apache.kafka.common.security.kerberos.NoMatchingRule: Non-simple name XYZ...@DOMAIN.COM after auth_to_local rule RULE:[1:$1@$0]

        at org.apache.kafka.common.security.kerberos.KerberosRule.apply(KerberosRule.java:190)

        at org.apache.kafka.common.security.kerberos.KerberosShortNamer.shortName(KerberosShortNamer.java:93)

        at org.apache.kafka.common.security.authenticator.DefaultKafkaPrincipalBuilder.applyKerberosShortNamer(DefaultKafkaPrincipalBuilder.java:131)

        ... 13 more

I have attached the server.properties file for reference.

Apologies for the long post, any help is greatly appreciated.

[Alf Baez]

Hi Jitu,

Have you fixed the issue?

Have you revised the guide again it's been updated just a day ago...

check all your prerequisites against:

...

Prerequisites

An LDAP server (e.g Active Directory) must be set up before starting up the Kafka cluster. The example below assumes that you have an LDAP server at the URL LDAPSERVER.EXAMPLE.COM:3268 that is accessible using DNS lookup from the host where the broker is run. The example expects a Kerberos-enabled LDAP server and the LDAP Authorizer configuration uses GSSAPI for authentication. These security settings and other configuration options must match your LDAP server configuration.

The example uses the following host, realm and port, these should be updated to point to your LDAP server.

• LDAP server host: LDAPSERVER.EXAMPLE.COM
• LDAP realm: EXAMPLE.COM
• LDAP port: 3268 (global context port used in the example)

At least one group must be created containing one or more users. The example assumes that your LDAP server contains a group named Kafka Developers and a user named alice who is a member of Kafka Developers group. The user principal and group must be updated to match the user and group from your LDAP server that you want to use for the tests

....