Casting to GenericContainer instead of GenericRecord in AvroRecordWriterProvider.java - line 50

126 views
Skip to first unread message

vikra...@openx.com

unread,
Aug 29, 2013, 5:24:30 PM8/29/13
to camu...@googlegroups.com
I was plugging in my own message decoder which was interpreting the incoming message to a specific Avro record object, however due to this particular casting in com/linkedin/camus/etl/kafka/common/AvroRecordWriterProvider.java , line 50 , to a GenericRecord its failing due to the fact that a specific Avro record object is not a sub-class of the GenericRecord interface.

I modified the code on line 50 to actually cast it to (GenericContainer) , which is a top level avro class that is above both GenericRecord and IndexedRecord(the super-class of specific record) and it seems to work fine. The only reason for this casting at present seems to be to get the underlying avro SCHEMA object.

Can you guys kindly make this change and push it. 

Felix GV

unread,
Aug 29, 2013, 5:35:00 PM8/29/13
to vikra...@openx.com, camu...@googlegroups.com
I'm curious to hear about the implications of such change...

For example, if I remember correctly (and please pardon me if my memory fails me), Ken said at some point that schema evolution only works with GenericRecord. So would it break decoder implementations that are capable of schema evolution if Camus was to introduce such change?

--
Felix


--
You received this message because you are subscribed to the Google Groups "Camus - Kafka ETL for Hadoop" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camus_etl+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Ken Goodhope

unread,
Aug 30, 2013, 1:03:06 AM8/30/13
to Felix GV, vikra...@openx.com, camu...@googlegroups.com
GenericContainer would provide the greatest flexibility.  Especially if you want to allow for non-record avro datums.   As for schema evolution, that happens mainly on the read side so there shouldn't be any issues.  Also, SpecificRecord punts to GenericRecord when it can't find a class definition, so from that angle it works too.

Ken
Reply all
Reply to author
Forward
0 new messages