Confluent 3.2.2 - Avro Consumer not able to consume Avro messages

[karan alang]

hi All 

- i've a java consumer program which is publishing Avro messages to Kafka Topic.

(Attached is the program.)

When i start the console Kafka consumer(kafka-avro-console-consume), i get an error as shown below 

Any ideas on what the issue is & how to resolve this ?

 $CONFLUENT/bin/kafka-avro-console-consumer --topic AvroTest --bootstrap-server nwk2-bdp-kafka-04.gdcs-qa.apple.com:9092 --from-beginning --new-consumer
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.5.3.0-37/confluent-3.2.2/share/java/kafka-serde-tools/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.5.3.0-37/confluent-3.2.2/share/java/schema-registry/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Processed a total of 1 messages
[2017-08-08 22:52:31,824] ERROR Unknown error when running consumer:  (kafka.tools.ConsoleConsumer$:105)
org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1
Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!
[2017-08-08 22:52:31,824] ERROR Unknown error when running consumer:  (kafka.tools.ConsoleConsumer$:105)
org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1
Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!

[Gwen Shapira]

It looks like you did your own serialization? Our Avro console consumer works with records written using Kafka Avro Serializer. The error is about a missing "magic byte" because our serializer adds a magic byte there and the console consumer expects it.

Our Avro serializers uses the Confluent Schema Registry to help you track schemas, share them between apps and validate compatibility when you make schema changes.

You can check out examples here:
https://github.com/gwenshap/kafka-examples/tree/master/AvroProducerExample/src/main/java/com/shapira/examples/producer/avroclicks
https://github.com/confluentinc/examples/tree/3.3.0-post/kafka-clients/specific-avro-producer

Gwen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/154b78a2-8d72-4c39-8124-d219e8133ea0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[karan alang]

Thanks, Gwen .. will check this.

regds,

Karan Alang

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/154b78a2-8d72-4c39-8124-d219e8133ea0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Confluent Platform" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/confluent-platform/HhiJaXUOEAU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CAA9QSJ%2BHMs-R2LpgYVXTvPTPrCh8h4Ax-sZKPESXX-M5xPmXJA%40mail.gmail.com.