Avro: Is is possible to serialize multiple objects to a file?

417 views
Skip to first unread message

Tom van den Berge

unread,
Jan 5, 2017, 9:06:24 AM1/5/17
to jackson-user
One common use case when using Avro is to serialize multiple objects to a single file. The schema will be included once.

Is there a way achieve this with jackson-dataformat-avro? 

If I understand it correctly, each time the AvroMapper serializes an object, the schema is included. And I couldn't find a way to combine multiple objects into a single file or output stream.

Regards,
Tom

Tatu Saloranta

unread,
Jan 24, 2017, 10:43:36 PM1/24/17
to jackson-user
Apologies for a slow response here: yes, once this:

https://github.com/FasterXML/jackson-dataformats-binary/issues/35

gets into release (2.7.9 and 2.8.7) this will work using
`SequenceWriter`, like so:

SequenceWriter w = mapper.writer(schema)
.writeValues(file);
w.write(item1);
w.write(item2);
// and so on
w.close();

and similarly `readValues()` (and resulting `MappingIterator`) allows
reading of such value sequences.
And I also fixed some issues with root values themselves; formerly
Maps did not work as root values, or scalars, now they are valid as
well.

-+ Tatu +-

ps. I did actually release micro-release 2.7.8.1 for Avro module which
has this fix, if anyone wants to test it before full 2.7.9 is out.
> --
> You received this message because you are subscribed to the Google Groups
> "jackson-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jackson-user...@googlegroups.com.
> To post to this group, send email to jackso...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages