Rehydrate kafka from secor backup

[Ankur Chauhan]

Hi all,

I am looking at secor as a way to take running backups of kafka topics and I was wondering if anyone has ideas on implementing or has pointers to an existing utility to "replay" the saved backup and reconstitute the kafka records?

[Pawel Garbacki]

You may want to use the OffsetMessageParser (secor.message.parser.class=com.pinterest.secor.parser.OffsetMessageParser) to partition the data on offsets. This will make replaying the logs easier. Writing a kafka producer to go over the logs and inject them into Kafka should be trivial.

On Wed, Apr 22, 2015 at 1:29 PM, Ankur Chauhan <an...@malloc64.com> wrote:

Hi all,

I am looking at secor as a way to take running backups of kafka topics and I was wondering if anyone has ideas on implementing or has pointers to an existing utility to "replay" the saved backup and reconstitute the kafka records?

--
You received this message because you are subscribed to the Google Groups "secor-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to secor-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[prasann...@gmail.com]

We jut have a separate indexer for those files for time and we use that for granular replays.