Cannot Consume Messages from Topic with ACL, Producer does not have problems

[Jay]

From the documentation (http://docs.confluent.io/3.1.1/kafka/authorization.html)

• consume from a topic, the principal of the consumer will require the READ operation on the topic and group resources.

And I have the following ACL:

Current ACLs for resource `Group:~`:

        User:CN=Reijay,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown has Allow permission for operations: Describe from hosts: *

        User:CN=Reijay,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown has Allow permission for operations: Read from hosts: *

Current ACLs for resource `Topic:TestAcl1`:

        User:CN=Reijay,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown has Allow permission for operations: Describe from hosts: *

        User:CN=Reijay,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown has Allow permission for operations: Read from hosts: *

        User:CN=Reijay,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown has Allow permission for operations: Write from hosts: *

Current ACLs for resource `Cluster:kafka-cluster`:

        User:CN=Reijay,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown has Allow permission for operations: Create from hosts: *

But when I tried to consume messages from topic TestAcl1, I got the following error:

ERROR Unknown error when running consumer: ( kafka.tools.ConsoleConsumer$)

org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: console-consumer-71033

Do not have problem with producing message, just the consumer. What other permissions do I need to set?

Thanks,

Reijay

[saras...@gmail.com]

I have the problem as well. I am able to produce the message but not able to consume it. While consuming the messages, I am getting the below error "org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: console-consumer-61109"

Here is more information

I am SSL setup based on the CERT

in server.properties:

super.users=User:<MYHOST_CERTNAME>
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer

ACLs:

Current ACLs for resource `Topic:sankara.topic.12`:
     User:CN=kafkatt2 has Allow permission for operations: Describe from hosts: *
    User:CN=kafkatt2 has Allow permission for operations: Read from hosts: *
    User:CN=kafkatt1 has Allow permission for operations: Describe from hosts: *
    User:CN=kafkatt1 has Allow permission for operations: Write from hosts: *

Error I am getting while consuming the message:

[2017-04-05 13:29:56,349] ERROR Unknown error when running consumer:  (kafka.tools.ConsoleConsumer$)
org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: console-consumer-61109

Information in "kafka-authorizer.log":

[2017-04-05 13:39:35,296] DEBUG operation = Describe on resource = Topic:sankara.topic.12 from host = 127.0.0.1 is Allow based on acl = User:CN=kafkatt2 has Allow permission for operations: Describe from hosts: * (kafka.authorizer.logger)
[2017-04-05 13:39:35,297] DEBUG Principal = User:CN=kafkatt2 is Allowed Operation = Describe from host = 127.0.0.1 on resource = Topic:sankara.topic.12 (kafka.authorizer.logger)

Thanks

Sankara

[Jay]

Hi Sankara,

I was able to resolved the issue.
See if you have access to the consumer-groups. In my case, I though I gave access to read from all consumer-groups, but turns out the command I ran did not gave such access.

So, try to add the --group=* to acl command that gave consumer access to the topic

(In my case, I run sudo /usr/bin/kafka-acls --authorizer-properties zookeeper.connect=<zookeeper_host:zookeeper_port> --add --allow-principal User:<cert_name> --topic <topic_name> --consumer --group=*)

[saras...@gmail.com]

Thanks Jay, it worked for me after granting access to --group=*

./bin/kafka-acls --authorizer-properties zookeeper.connect=<ZK> --add --allow-principal User:CN=kafkatt2 --consumer --topic sankara.topic.12 --group=*

What should I if I want to grant access to specific group instead of "*"

Earlier I what I had was

./bin/kafka-acls --authorizer-properties zookeeper.connect=<ZK> --add --allow-principal User:CN=kafkatt2 --consumer --topic sankara.topic.12 --group sankaratest

Thanks

Sankara