Schema Registry Error: Conversion of JSON to Avro failed with complex schema

[Nguyen Cao]

Hi,

I am running into a weird error related to  Conversion of JSON to Avro failed with complex schema. Let's me explain in details below.

I have defined a record schema as follow:

{

    "topic": "PageView",

    "format": "avro",

    "partition":0,

    "group":"pageviewConsumerGroup",

    "value-schema":{

    "type":"record",

    "name":"PageView",

    "fields":[

    {"name":"createdAt","type":"string"},

                {"name":"referrer","type":"string"},

    {"name":"localCreatedAt","type":[{

          "type":"record",

          "name":"LocalTime",

          "fields":[

            {"name":"year","type":"int"},

            {"name":"month","type":"int"},

            {"name":"week","type":"int"},

            {"name":"day","type":"int"}

          ]

          },"null"]

        },

    {"name":"url","type":"string"},

    {"name":"sessionId","type":"string"},

    {"name":"merchantSiteId","type":"string"},

        {"name":"pageCategory","type":["string","null"]},

        {"name":"pageCategoryPath","type":["string","null"]},

        {"name":"firstInCategory","type":"boolean"}

]

}

}

Basically I defined a complex record type which contains another record in one of its field (localCreatedAt)

Now when I send the following message complied with the above schema, I always get the error: APIError: HTTP 422 Conversion of JSON to Avro failed.

{ url: 'http://demo.divvit.com/shoes/product-a.html',

  referrer: 'http://demo.divvit.com/',

  createdAt: '2015-12-22T11:39:00.000Z',

  localCreatedAt: { year: 2015, month: 11, day: 22, week: 52 },

  sessionId: '56793706ef39231b00bc3300',

  merchantSiteId: '567922c7b8087f1b00e5d822',

  pageCategory: null,

  pageCategoryPath: null,

  firstInCategory: true }

However, if I send the above message with a little change of localCreatedAt to null then the error disappears, message sent.

Btw, I am using nodejs client provided by Confluent.

Thanks,

Nguyen

[Nguyen Cao]

For your better references with my example, I did see the PageView-value subject successfully registered with the schema registry via accessing its REST API.

One more thing is that with the nested record type (in my case which is LocalTime), should it be also processed to register as a subject of schema registry also? If so, then later we can reference these already defined record type within other schema, right?

Look forward to have your thought,

Thanks,

Nguyen 

[Ewen Cheslack-Postava]

Nguyen,

The first thing that stands out to me is that localCreatedAt is a union type of LocalTime or null. The JSON encoding for union types requires a "wrapper" object for non-null values, so I think you'd need:

"localCreatedAt": {"LocalTime": { year: 2015, month: 11, day: 22, week: 52 }}

for that field. See https://avro.apache.org/docs/1.7.7/spec.html#json_encoding for more details on the JSON encoding of Avro.

-Ewen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/50494d78-80fb-4c72-97ed-cb64a1dd8b1a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Ewen

[Nguyen Cao]

Ewen, you just point me to the part I skipped.

Problem solved now. Many thanks. 

On Tuesday, December 22, 2015 at 11:06:12 PM UTC+7, Nguyen Cao wrote: