Unknown magic byte!

[sraa]

Hi all,

When I consume from Kafka broker after a while consumer continuously throwing the following exception.

2018-06-21 13:07:41.885 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] ERROR o.s.k.listener.LoggingErrorHandler - Error while processing: null

org.apache.kafka.common.errors.SerializationException: Error deserializing key/value for partition trip-0 at offset 2638. If needed, please seek past the record to continue consumption.

Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1

Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!

I can't understand what going on. if anyone can help that would be great.

Kafka Version - 1.1.0

Avro Version - 1.8.2

Confluent Version - 4.1.0 

Thank You.

[Robin Moffatt]

Usually that means that you've got data on the topic that is not valid Avro. Maybe use something like kafkacat to inspect the message in question and see? 

-- 

Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/36213fdb-d7d9-4117-ada8-518d2da766cf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Otto]

Your message might be valid Avro, but not valid Confluent-Avro.  The Confluent platform expects that binary Avro messages in kafka are prefixed with a magic byte and an integer schema id.

On Thu, Jun 21, 2018 at 6:12 AM, Robin Moffatt <ro...@confluent.io> wrote:

Usually that means that you've got data on the topic that is not valid Avro. Maybe use something like kafkacat to inspect the message in question and see? 

-- 

Robin Moffatt | Developer Advocate | ro...@confluent.io | @rmoff

On Thu, 21 Jun 2018 at 08:47, sraa <lah...@pickme.lk> wrote:

Hi all,

When I consume from Kafka broker after a while consumer continuously throwing the following exception.

2018-06-21 13:07:41.885 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] ERROR o.s.k.listener.LoggingErrorHandler - Error while processing: null

org.apache.kafka.common.errors.SerializationException: Error deserializing key/value for partition trip-0 at offset 2638. If needed, please seek past the record to continue consumption.

Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1

Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!

I can't understand what going on. if anyone can help that would be great.

Kafka Version - 1.1.0

Avro Version - 1.8.2

Confluent Version - 4.1.0 

Thank You.

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/36213fdb-d7d9-4117-ada8-518d2da766cf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Confluent Platform" group.

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/CA%2BJsER0uVHxfW0Tskt8T%3Dt4qTY6g44mO3dXWFg40Nspgr8_geQ%40mail.gmail.com.