Using tracking event processors with JdbcTokenStore

[Michał Duda]

Hi,
I'm having trouble configuring tracking event processor sin my application. I'm using Spring Boot and want to use JdbcTokenStore. I'm using SimpleEventBus and my app works nicely with default subscribing event processors. What I already did:
1. Created a TokenEntry table
2. Enabled tracking processors by doing:

@Autowired
public void configure(EventHandlingConfiguration config, EventStore eventStore) {
  config.usingTrackingProcessors();

3. Set up JdbcTokenStore:

@Bean
public TokenStore jdbcTokenStore(@Qualifier("eventStoreDS") DataSource dataSource, Serializer serializer)
    throws SQLException {
  return new JdbcTokenStore(new SpringDataSourceConnectionProvider(dataSource), serializer);
}

4. Added a couple of @ProcessingGroup("...") over my event listeners.

The result is that token entries are inserted into TokenEntry but the 'token' and 'tokenType' columns are 'null' for all of them after processing the events. Anything else I need to do besides what I've already done?

Thanks!

[Allard Buijze]

Hi Michal,

the only way I can reproduce getting null values for token and tokenType, is when the processor hasn't handled any events. As soon as the first message is processed, the token becomes non-null.

How does the rest of your configuration look?

Cheers,

Allard

--
You received this message because you are subscribed to the Google Groups "Axon Framework Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to axonframewor...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michał Duda]

Hi Allard,
I'm using an explicitly declared EmbeddedEventStore with a custom EventStorageEngine (very similar to JdbcEventStorageEngine with some additional info)  and an explicit SimpleEventBus. My event handlers are just Spring beans with @EventHandler on their methods. You said that these values are null only when the processor hasn't handled any events but my events are processed by the TrackingEventProcessor, event handlers execute and projections are updated, it's only those two columns in the token store that are not updated. One thing that I noticed is that if I explicitly register the tracking processors by doing something like this:

@Autowired
public void configure(EventHandlingConfiguration config, EventStore eventStore) {
  config.usingTrackingProcessors();

  config.registerTrackingProcessor("projection 1", conf -> eventStore);
  config.registerTrackingProcessor("projection 2", conf -> eventStore);
}

the tokens start being properly updated in the token store.

[Allard Buijze]

Hi Michal,

I think the problem is the fact that you register both an Event Store and an Event Bus. In Axon 3, the Event Store is a replacement for the Event Bus. It's not unlikely that your processors are reading from the Event Bus, which gives <null> tokens (as it doesn't retain old events anyway).

To resolve this, simply remove the EventBus from your configuration.

Cheers,

Allard

[Michał Duda]

Unfortunately in our situation we have to have a separate Event Store and Event Bus. I just checked and EventHandlingConfiguration.usingTrackingProcessors() uses the Event Bus. Axon provides the ability to register tracking event processors using EventStore as source but looks like you have to register each processor explicitly either programmatically or in application.yml. Perhaps it would make sense to be able to specify the default source when using EventHandlingConfiguration.usingTrackingProcessors()?

[Allard Buijze]

Hi,

the 3.0.4 release of last week provides an extra option in the Configuration API where you can explicitly define the source of events for each processor (tracking or subscribing), that you create. They all default to 'the' Event Bus/Store. I'm not sure what happens when there are 2 instances available.

Cheers,

Allard

[Michał Duda]

Thanks Allard,

I have two more questions regarding using tracking processors:

1) What is the purpose of 'segment' in tracking tokens? It isn't described anywhere is the docs. The only 'segment' concept I found in the docs is related to the command side.

2) Is it possible to assign processing groups to event handlers dynamically or have the @ProcessorGroup name be pulled from the configuration/application.yml/environment variable? The use case I have for this is the replay takes a long time so I'd like to perform it on the second instance of projection. If I just rename the processing group that my event handlers are assigned to and start the app, the replay should happen automatically.

Kind regards,
Michal

[christoph...@trifork.nl]

On Tuesday, May 16, 2017 at 5:37:52 PM UTC+2, Michał Duda wrote:

Thanks Allard,

I have two more questions regarding using tracking processors:

1) What is the purpose of 'segment' in tracking tokens? It isn't described anywhere is the docs. The only 'segment' concept I found in the docs is related to the command side.

Hey Michal, please allow me to respond to your first question (only) , as I am working on segments in the tracking processors. Currently (Axon 3.0.4) has only the root segment 0. The TokenStore is already enabled for segmentation. The basic idea is that we can segment the stream of events to be processed in parallel by multiple TrackingEvenProcessors, enhancing the overall event throughput. If you would like learn more about this work, please check this issue 221 and the PR which goes with it. 

[Allard Buijze]

Hi Michal,

I'll answer your second question, then. You can assing handlers to processors using a function, giving you all flexibility you want.
Check out the EventHandlingConfiguration class. In Spring, you can create an @Autowired annotated method that accepts the EventHandlingConfiguration as parameter to change the configuration.

Cheers,

Allard

[Michał Duda]

Thank you both for help! EventHandlingConfiguration.assignHandlersMatching looks like what I need. I'll also have a look at the segment PR.

Cheers,
Michal