Using Onyx for detecting inactivity

[Ivan Valchev]

Hi there,

I would like to know how can I use Onyx to solve the following problem.

Say we have client application that when used would communicate with a remote server. All messages send by the client application are placed on a stream (like Kafka). What I would like to achieve is detect when a client has become inactive which in my case would mean that there was no message send by the client in the last 15 mins. When that occurs I would like to have a message placed on a stream indicating that. I would also want to emit this inactivity indication for downstream processing in a workflow as well.

I believe this could be modeled with Onyx windows and state management support but not quite sure how to emitting the state should happen. Any help will be much appreciated.

[Ivan Valchev]

My initial guess would be to use global windows with trigger timer period set to 15 mins and refinement set to discarding. Is there a possibility to instruct the trigger to fire when a window extent ends that way you could use fixed window instead of a global one?

[Mike Drogalis]

This one is a little unintuitive to think about with Onyx windows because our state API is capable of handling events received out-of-order. That is, we can correctly handle data send with events whose timestamps are received in an order in which they did not occur.

Given that windows anticipate receiving data from the (potentially very long ago) past, "timing out" a window isn't something that's support directly from the API. Using the 15 minute fixed windows approach doesn't work well because we assume each 15 minute window could receive data from any timestamp at any point in the stream. Session windows are closer to what you want, but they don't have a notion of timing out due to inactivity either. They only expand.

I would recommend Session windows (with session key user ID), and a timer trigger. The trick here is that each time the trigger fires, you need to compare "this state" with the state from the previous trigger. We currently don't keep the previous state around to do a diff, but I'm planning on adding a patch for this soon since I've encountered the same problem. For the moment, you can place an atom in the event map and swap the state into it's contents. This basically let's you keep around the previous state to do the comparison.

If you have both the current state and the previous state, you can determine which users haven't had any activity based off the wallclock time of the machine. This has the obvious downside of requiring that the wallclock of the peer is close enough to being correct, but this problem inherently involves wallclock time since all data is presumed to be received in order.

Happy to elaborate more if needed.

--
You received this message because you are subscribed to the Google Groups "Onyx" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onyx-user+unsubscribe@googlegroups.com.
To post to this group, send email to onyx...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/onyx-user/ab585cc6-d616-46b3-a1cf-ef166dc0de29%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Ivan Valchev]

Hi Mike,

Thanks a lot for the quick reply. For my use case I consider the time of the event to be equal to the time it is received by the server.

If I understand you correctly the trick here is to detect whether any user events have occurred between two trigger executions. When would the timer trigger stop firing for a given session window? It could be the case the user uses the application very rarely or even only once.

[Mike Drogalis]

In that case, you make make a task before your window to tack on the current wallclock time as a key into your segment. Timers fire on recurring intervals regardless of whether or not activity has occurred, so they would never stop firing.

[Ivan Valchev]

I see okay, one last thing I would like to ask you. Is it possible to configure a trigger to fire at window extend end? From the documentation for :trigger/on it is not clear to me how this can be achieved.

And Mike, thank you for the answers and your time :) Keep up the good work!

[Mike Drogalis]

Onyx never presumes that it's seen all the data before some arbitrary time, say 04-10-2017 midnight GMT. Thus a window extent can never "end". We're always presuming more data may come in from the past.

No problem!