AvroConverter fromConnectData not serializing ?

[Sreejith S]

Hi All,

I have schema registry up and running and set below configuration in connect-standalone.properties.

key.converter=io.confluent.connect.avro.AvroConverter
value.converter=io.confluent.connect.avro.AvroConverter

key.converter.schema.registry.url=http://localhost:8081
value.converter.schema.registry.url=http://localhost:8081

In my Source Task i created the Schema and struct for a simple schema like below

Schema conenctSchema = SchemaBuilder
        .struct()
        .name("test")
        .field("name", org.apache.kafka.connect.data.Schema.OPTIONAL_STRING_SCHEMA)
        .field("loc", org.apache.kafka.connect.data.Schema.OPTIONAL_STRING_SCHEMA)
       .build();
 
Struct msgStruct = new Struct(conenctSchema);
 msgStruct.put("name", "sreejith");
 msgStruct.put("loc", "IN");

and passing the source record as

 SourceRecord sourceRecord = new SourceRecord(null, null, "schema-test-new", conenctSchema,msgStruct);

My connector is running with out any exceptions , but when i consume it using kafka-avro-consumer its like 

null    {"name":{"string":"sreejith"},"loc":{"string":"IN"}}

also when i am consuming from normal kafka-console-consumer , its printing like this ,

 ▒sreejithIN

Why the message is not converted in to Bytes ? Why its not in MAGIC_BYTE ID MSG IN BYTES format ? Am i missing something ?

Thank You

Sreejith

[Ian Wrigley]

The data is stored as serialized bytes in Kafka. When you consume using kafka-avro-console-consumer, the Consumer uses the Schema Registry to work out how to deserialize the data. It displays the output in a JSON format just because that’s a convenient way to show the output. By default, kaka-console-consumer just uses ByteArrayDeserializer; it’s outputting the bytes to the console, but of course since the fields were strings, the binary version of the data is human-readable.

Ian.

---
Ian Wrigley
Director, Education Services

Confluent, Inc

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/4d01d5e3-b762-49ab-afe2-103b90bdd4d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sreejith S]

Thank you Ian.  Clarified.. 

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/4d01d5e3-b762-49ab-afe2-103b90bdd4d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Confluent Platform" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/confluent-platform/XW2W4vQVnv8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/78FD28E0-99B0-48BE-BAA8-6E8AA1CDD907%40confluent.io.