Does Kamon handle stash() / unstash() properly?

[Eax Melanhovich]

Hello.

Lets say there are 5 messages in actors queue and 10 messages in actors
stash. What number of messages will be reported by Kamon to Datadog or
StatsD?

--
Best regards,
Eax Melanhovich
http://eax.me/

[Ivan Topolnjak]

Hello Eax, 

I'm glad that you mentioned this as I just realized that you will get some sort of unexpected metrics when messages are stashed: When a message arrives for the first time it is measured as any normal message would, meaning that the time-in-mailbox and processing-time metrics are recorded. If the message is stashed then the whole Envelope is kept in the stash and once unstashed then the messages are put back into the mailbox, meaning that they will go again through the normal processing path and the time-in-mailbox and processing-time metrics will be recorded again, but this time, since the envelope is the same as the initial one then the time-in-mailbox that gets recorded is not counting since the message was stashed or put back in the queue, but rather since it was initially put in the actor's mailbox. 

To me, that time-in-mailbox measurement is really unexpected, specially because it is very likely that stashed-time will have a different behavior to actual time-in-mailbox for a given actor and mixing both kinds of recordings in the same metric will render it useless.. now, how to solve this issue? my current idea would be add a new metric called "time-in-stash" that would reflect exactly what it says and additionally reset the capture time on the envelope so that once a message is unstashed and put back into the mailbox the time-in-mailbox will again reflect exactly what it is, the time in the mailbox :)... how that sounds to you?

I'm not a fan of using Stash so I don't have real world experience on patterns in which people use it, but I guess that the proposed solution would be a decent one. Hope to hear your feedback, regards!

[Eax Melanhovich]

Hello, Ivan.

Sounds reasonable :)

[Ivan Topolnjak]

Cool, will be tracking this improvement here: https://github.com/kamon-io/Kamon/issues/252