Trigger fired on session window closed?

[nacho...@fundingcircle.com]

Hello,

We are trying to use Onyx to buffer a series of events match them by a particular property and split the stream in matched and unmatched after the buffering time expires. 

We have tried to do this with a session window but we are not able to identify with a trigger when a particular session window expires. Is it possible to do this with the current implementation? 

[Mike Drogalis]

Sessions never "expire" in the sense that Onyx doesn't assume your data is coming in any particular order. That is, you might receive data later on that extends a session, but you didn't see it earlier due to a condition inducing stream disorder.

You'd need to have a trigger identify a piece of information that indicates that your window is closed - e.g. a Punctuation trigger. In the absence of that, you can set a Timer trigger that periodically checks for "expired" windows - whatever that means for your exact use case. We're talking in Gitter about adding a finer-grained discarding mode, which I think would help out here. Does that make sense, or would you still be stuck for your problem?

--
You received this message because you are subscribed to the Google Groups "Onyx" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onyx-user+...@googlegroups.com.
To post to this group, send email to onyx...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/onyx-user/181c7929-566b-474d-a8f5-f6882da78352%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[nacho...@fundingcircle.com]

Thanks a lot for your reply :) 

We already tried this approach associating a timer trigger with a period almost matching the session window time-gap. We were hoping to get a trigger execution matching our buffer time for the data in the window. It also matches the selected time gap. 

The behaviour we have observed is that the trigger is fired: 

  1) when the first event arrives at the stream and opens a new session window

  2) when the timer timeouts 

  3) and then periodically even if no other matching segments for the session key arrive in the stream to extend the window session. 

We are not expecting invocation number either 1) or 3) Probably because of our misunderstanding of the way Onyx works. With this behaviour we can implement the solution by using an external key-value store to disambiguate the sequence of trigger executions but having some additional trigger policies or a way of filtering the status of the window would make things easier. 

Thanks again

[Mike Drogalis]

This is the expected behavior for a Timer trigger. Can you please open an issue against Onyx core to suggest a new trigger type for the behavior you suggested? I think it's reasonable behavior in an alternate trigger if the new refinement modes that we're looking at don't cover this.

To view this discussion on the web visit https://groups.google.com/d/msgid/onyx-user/0df1fc74-ab55-4042-9c16-6242be8d63ab%40googlegroups.com.

[nacho...@fundingcircle.com]

Will do! 

Thanks a lot :)