Initializing a tuple with data based on its fields grouping

[paces...@gmail.com]

I have a bolt that counts incoming tuples grouped by field and periodically (via a tick tuple) emits those counts. I need this bolt to emit counts for all of the possible values that it might receive via its fields grouping which also are in our database. So the bolt must be initialized with counters for all of the tuples it might receive. The trick is knowing which tuples each bolt might receive since it uses a fields grouping. One solution is to have another bolt upstream that sends these initialization tuples to my bolt. But if my bolt is restarted then these counters will be lost. Are there any other solutions for this?

Thanks,

Josh

[Philip O'Toole]

IMHO to solve a problem like this you need to store the values outside of Storm. Use something Iike Cassandra, with its counter functionality. Then each bolt doesn't need to know initial values on startup, it just needs to send "increment" statements to the database. 

Philip

--
You received this message because you are subscribed to the Google Groups "storm-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[paces...@gmail.com]

Thanks for the response Phillip,

The initial state that the bolt gets isn't actually counts, it's whether or not a particular ID that it might receive tuples for is eligible for being counted. The problem I'm having is getting this initialization data to the right bolts upon startup.

- Josh