IllegalArgumentException: Unable to subscribe to the Kafka topic _schemas backing this data store

[Binoy Thomas]

Getting this message when turning on ACLs. However not an issue with just SSL turned on.

server.properties ACL configuration is as below. Since the application is running locally both the Kafka Server & Schema Registry have certificates that have the CN=localhost

############################# ACL Configs #############################

authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer

#allow.everyone.if.no.acl.found=true

super.users=User:CN=localhost

############################# Log Basics #############################

Since I wanted grant all privileges to the Kafka broker nodes ran the below. This indirectly will also provide the Schema Registry with such privileges (as this is a local env)

/usr/local/kafka/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:CN=localhost --operation ALL --topic * --group * --cluster *

Starting up Schema Registry (after Zookeper & Kafka) gave the below error message

[2017-06-30 11:08:21,440] ERROR Server died unexpectedly:  (io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain:51)

java.lang.IllegalArgumentException: Unable to subscribe to the Kafka topic _schemas backing this data store. Topic may not exist.

at io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread.<init>(KafkaStoreReaderThread.java:126)

at io.confluent.kafka.schemaregistry.storage.KafkaStore.init(KafkaStore.java:162)

at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.init(KafkaSchemaRegistry.java:204)

at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.setupResources(SchemaRegistryRestApplication.java:55)

at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.setupResources(SchemaRegistryRestApplication.java:37)

at io.confluent.rest.Application.createServer(Application.java:149)

at io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain.main(SchemaRegistryMain.java:43)

So then explicitly ran the below. I would have thought the earlier command should have granted access to all topics.

/usr/local/kafka/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:CN=localhost --operation ALL --topic _schemas

But this did not help either....

[Gwen Shapira]

You probably want to enable the Authorizer log for debugging. It will tell you which principal was denied and why.

For instructions, go here: http://docs.confluent.io/current/kafka/authorization.html?highlight=acl#command-line-interface and scroll way down.

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/f7debd46-1436-4729-81fd-ba1abfcd1878%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Binoy Thomas]

Thanks Gwen - 

I'm getting the below error. Shouldn't --operation ALL cover this error?

2017-06-30 12:09:52,381] DEBUG Principal = User:CN=localhost,OU=Tech,O=Pieces Tech,L=Dallas,ST=TX,C=US is Denied Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger)

[2017-06-30 12:09:53,390] DEBUG Principal = User:CN=localhost,OU=Tech,O=Pieces Tech,L=Dallas,ST=TX,C=US is Denied Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger)

[2017-06-30 12:09:54,395] DEBUG Principal = User:CN=localhost,OU=Tech,O=Pieces Tech,L=Dallas,ST=TX,C=US is Denied Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger)

[2017-06-30 12:09:55,404] DEBUG Principal = User:CN=localhost,OU=Tech,O=Pieces Tech,L=Dallas,ST=TX,C=US is Denied Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger)

[2017-06-30 12:09:56,410] DEBUG Principal = User:CN=localhost,OU=Tech,O=Pieces Tech,L=Dallas,ST=TX,C=US is Denied Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger)

[2017-06-30 12:09:57,414] DEBUG Principal = User:CN=localhost,OU=Tech,O=Pieces Tech,L=Dallas,ST=TX,C=US is Denied Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger)

[2017-06-30 12:09:58,418] DEBUG Principal = User:CN=localhost,OU=Tech,O=Pieces Tech,L=Dallas,ST=TX,C=US is Denied Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger)

[2017-06-30 12:09:59,426] DEBUG Principal = User:CN=localhost,OU=Tech,O=Pieces Tech,L=Dallas,ST=TX,C=US is Denied Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger)

[2017-06-30 12:10:00,430] DEBUG Principal = User:CN=localhost,OU=Tech,O=Pieces Tech,L=Dallas,ST=TX,C=US is Denied Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger)

[2017-06-30 12:10:01,434] DEBUG Principal = User:CN=localhost,OU=Tech,O=Pieces Tech,L=Dallas,ST=TX,C=US is Denied Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas (kafka.authorizer.logger)

I then also when and explicitly added all hosts and DESCRIBE operation as well.

/usr/local/kafka/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --list

Current ACLs for resource `Group:controller.log`:

  User:CN=localhost has Allow permission for operations: All from hosts: controller.log

Current ACLs for resource `Topic:AWS-Personal`:

  User:CN=localhost has Allow permission for operations: All from hosts: *

Current ACLs for resource `Cluster:kafka-cluster`:

  User:CN=localhost has Allow permission for operations: All from hosts: controller.log

User:CN=localhost has Allow permission for operations: All from hosts: *

Current ACLs for resource `Topic:controller.log`:

  User:CN=localhost has Allow permission for operations: All from hosts: controller.log

Current ACLs for resource `Group:AWS-Personal`:

  User:CN=localhost has Allow permission for operations: All from hosts: *

Current ACLs for resource `Topic:_schemas`:

  User:CN=localhost has Allow permission for operations: Describe from hosts: controller.log

User:CN=localhost has Allow permission for operations: All from hosts: controller.log

User:CN=localhost has Allow permission for operations: All from hosts: *

[Gwen Shapira]

I think the user doesn't match:

User:CN=localhost,OU=Tech,O=Pieces Tech,L=Dallas,ST=TX,C=US

Is not same as User:CN=localhost

We discuss customization a bit here: http://docs.confluent.io/3.2.0/kafka/authorization.html#further-configuration

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/49dac074-ebfc-4056-9259-25ff18458057%40googlegroups.com.

[Binoy Thomas]

Perfect.... its working now Gwen... thank you for pointing that out. I assumed that the user by default was the CN value.

On Friday, June 30, 2017 at 11:32:48 AM UTC-5, Binoy Thomas wrote: