Schema Registrt start fails with StoreInitializationException

[Rajasekar Elango]

I am trying to get  schema registry work with our installation of kafka brokers/zookeeper, but start up fails with error:

[2015-06-29 13:19:49,991] ERROR Error starting the schema registry (io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication:57)
io.confluent.kafka.schemaregistry.exceptions.SchemaRegistryInitializationException: Error initializing kafka store while initializing schema registry
    at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.init(KafkaSchemaRegistry.java:164)
    at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.setupResources(SchemaRegistryRestApplication.java:55)
    at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.setupResources(SchemaRegistryRestApplication.java:37)
    at io.confluent.rest.Application.createServer(Application.java:104)
    at io.confluent.kafka.schemaregistry.rest.Main.main(Main.java:42)
Caused by: io.confluent.kafka.schemaregistry.storage.exceptions.StoreInitializationException: io.confluent.kafka.schemaregistry.storage.exceptions.StoreException: Failed to write Noop record to kafka store.
    at io.confluent.kafka.schemaregistry.storage.KafkaStore.init(KafkaStore.java:151)
    at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.init(KafkaSchemaRegistry.java:162)
    ... 4 more
Caused by: io.confluent.kafka.schemaregistry.storage.exceptions.StoreException: Failed to write Noop record to kafka store.
    at io.confluent.kafka.schemaregistry.storage.KafkaStore.getLatestOffset(KafkaStore.java:363)
    at io.confluent.kafka.schemaregistry.storage.KafkaStore.waitUntilKafkaReaderReachesLastOffset(KafkaStore.java:220)
    at io.confluent.kafka.schemaregistry.storage.KafkaStore.init(KafkaStore.java:149)
    ... 5 more

It works fine when I start schema registry with kafka brokers bundled with confluent platform. Our installation has security features (SSL mutual auth) on top  latest 0.8.2 version of kafka. We know that all features of kafka works fine with our customized kafka, but not sure why schema-registry doesn't work. When I turned on debug logging on schema registry, I see it's trying to produce to _schemas topic in a loop, but gives up after some time.

Any hint on what could be the problem..? How can we trouble shoot further so that we get identify the root cause?

Thanks,
Raja.

[Geoffrey Anderson]

Hi Raja,

This looks like a connectivity issue between Schema Registry and the Kafka brokers. It may be related to the SSL patch, or there could be some a configuration issue (maybe it can't connect with using the broker metadata it is pulling from Zookeeper?)

A couple questions/ideas:

1) Do you know what client code you applied your SSL patches to? Perhaps you patched the old producer (kafka.producer), but not the new one (org.apache.kafka.clients.producer). If so, this may be the problem, since the Schema Registry uses the new producer to produce to the _schemas topic.

2) One way to check if it's related to producer code (as opposed to something in the Schema Registry) would be to try to run console producer using the jars in the confluent platform:

cd <confluent_platform>

bin/jkafka-console-producer --new-producer --broker-list <your_broker_list> --topic <a_topic>

3) You could also make sure there is no problem opening tcp connection to the brokers in your broker list from the machine where you're running the schema registry:

foreach broker:

  try:

    telnet <host> <port>

Let us know what clues you find if you try these ideas.

Thanks!

Geoff

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/183dd839-cc13-465c-ae12-932e5a79b6df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rajasekar Elango]

Thanks Geoffrey,

You hit nail on the head, yeah we patched only old producer and new producer doesn't work with SSL. Do you know if there is any easy way to make schema registry use old producer or is it designed to work with only new producer?

Thanks,

Raja.

--
You received this message because you are subscribed to a topic in the Google Groups "Confluent Platform" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/confluent-platform/-ihpA4XjsII/unsubscribe.
To unsubscribe from this group and all its topics, send an email to confluent-platf...@googlegroups.com.

To post to this group, send email to confluent...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CAM%3DixSKcSFE%3DY6iJ6i7QQbj%2BXcL9FpWNjVFReRbyzugegjDScQ%40mail.gmail.com.

[Geoffrey Anderson]

Hi Raja,

The schema registry takes advantage of some of the features of the new producer in its internal design, so there isn't a simple way to drop in the old producer.

Thanks,

Geoff

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CACcs26nLw0MGS6NUrjf6h2v2fX-DUgkJru6ZdvLPWTb8L_ytiw%40mail.gmail.com.