different results on UI using stats sample rate parameter

374 views
Skip to first unread message

Corne

unread,
Apr 18, 2012, 10:11:23 PM4/18/12
to storm-user
Hi

I ran a topology into a cluster and I made some tests setting the
"stats sample rate" parameter to 1 and I could see that the final
number of acked of the bolt is fine (lets say 1 million), so I made
the exact same test setting the parameter to 0.05 and the final number
of acked is less than the number I was expecting (1 million) or
sometimes it is grater, however I write my results into a log and I
can see that the actual number of tuples processed was 1 million (the
expected one), so :

is this an UI related issue?

Or do you know why is this happening?

Thanks

Nathan Marz

unread,
Apr 19, 2012, 7:04:32 PM4/19/12
to storm...@googlegroups.com
It's likely a UI-related issue. How much over/under 1 million are you seeing the numbers?
--
Twitter: @nathanmarz
http://nathanmarz.com

Cornelio Iñigo

unread,
Apr 19, 2012, 7:17:07 PM4/19/12
to storm...@googlegroups.com
Hi

Ok.
Some cases over/under 20, over/under 30, 60, 100 and in one case under 360

Thanks

2012/4/19 Nathan Marz <natha...@gmail.com>



--
Cornelio

Nathan Marz

unread,
Apr 19, 2012, 7:50:35 PM4/19/12
to storm...@googlegroups.com
OK, that's just the result of the fact that it's sampling. The way it works is that if you choose a sampling rate of 0.05, it will pick a random element of the next 20 events in which to increase the count by 20. So if you have 20 tasks for that bolt, your stats could be off by +-380. 

Cornelio Iñigo

unread,
Apr 19, 2012, 7:52:39 PM4/19/12
to storm...@googlegroups.com
Thanks for the quick response Nathan :)

2012/4/19 Nathan Marz <natha...@gmail.com>



--
Cornelio

Reply all
Reply to author
Forward
0 new messages