Replaying events with Kafka

[Gustavo Monti]

Hi,

I'm using Axon 3.3.5 with spring-boot and I'm trying to replay events in order to get databases sync. What I did was deleting my query db to get all the events from command db again.

My query side is configured with:

axon:

  serializer:

   general: XSTREAM

  eventhandling:

    processors:

      "[bankevents]":

        source: kafkaMessageSource

        mode: TRACKING

  kafka:

    default-topic: "events"

    consumer:

      group-id: "bank-group"

      bootstrap-servers:

        - localhost:9092

I tried to reset token in order to get it working, but nothing happens:

config.eventProcessorByProcessingGroup("bankevents", TrackingEventProcessor.class)

        .ifPresent(trackingEventProcessor -> {

            trackingEventProcessor.shutDown();

            trackingEventProcessor.resetTokens();

            trackingEventProcessor.start();

        });

Any advice?

[Sanjaya Kumar Sahoo]

Same issue for me, looking for suggestions

[Gustavo Monti]

I tried to delete token manually and didn't work too.

[Allard Buijze]

Hi,

there seems to be an issue with the KafkaTrackingToken in combination with how Kafka wants to coordinate consumers. This is being addressed at the moment.

Kind regards,

Allard

--
You received this message because you are subscribed to the Google Groups "Axon Framework Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to axonframewor...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Allard Buijze
CTO

E: allard...@axoniq.io
T: +31 6 34 73 99 89

[Gustavo Monti]

Hi Allard,

Thanks for your replay.

Can you send me the git issue? Maybe I can contribute with my logs and testing it.

[Gustavo Monti]

I tried the 3.4-SNAPSHOT today and the problem still persist there.

Can you send me the git issue? Maybe I can contribute with my logs and testing it.

[Gustavo Monti]

Any help here?

[Allard Buijze]

Hi Gustavo,

the issue isn’t fixed yet in the master branch. It has been contributed by an external developer, so we’re also dependent on his availability. I’ll ping him for the progress.

Cheers,

Allard

On Fri, 28 Sep 2018 at 02:20, Gustavo Monti <gust...@gmail.com> wrote:

Any help here?

--
You received this message because you are subscribed to the Google Groups "Axon Framework Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to axonframewor...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gustavo Monti]

Hello Allard,

It will be fixed with 4.0 version? Do you have the git issue, so I can track and contribute there.

[Allard Buijze]

Hi Gustavo,

in the Axon 4.0 release, we've moved the Kafka connector (as well as some other connectors) to different repositories. For Kafka, this is: https://github.com/AxonFramework/extension-kafka. There is currently one open issue, which covers the replay issue as well.

The KafkaConnector has not yet been released as 4.0, as we want this issue to be resolved first.

Cheers,

Allard

[cfu...@gmail.com]

So I am assuming this issue is fixed in version 4 as I am not seeing it in the issue list anymore.  I do have a question about this KafkaTrackingToken, is it just any offset of Kafka partition , or in case of multiple partitions on the topic, how is it achieving the tracking since Kafka does not guarantee ordering among different partitions.

[Allard Buijze]

The Kafka related issues have moved to the Kafka extention's repository:

https://github.com/AxonFramework/extension-kafka/issues

Ordering across partitions will not be guaranteed. Axon will do a best-effort ordering based on timestamp.

Hope this helps.

Cheers,

Allard Buijze

CTO

E: allard...@axoniq.io

T: +31 6 34 73 99 89

On Tue, 5 Feb 2019 at 18:46, <cfu...@gmail.com> wrote:

So I am assuming this issue is fixed in version 4 as I am not seeing it in the issue list anymore.  I do have a question about this KafkaTrackingToken, is it just any offset of Kafka partition , or in case of multiple partitions on the topic, how is it achieving the tracking since Kafka does not guarantee ordering among different partitions.

--

[cfu...@gmail.com]

Thanks for confirming this.  I totally understand the difficulties across partitions as this is Kafka's limitation.