Is outputCollector.emit ThreadSafe?

1,389 views
Skip to first unread message

Luis Darío Simonassi

unread,
Apr 7, 2012, 10:35:54 AM4/7/12
to storm-user
Hello all!
I've a MergeBolt class that has the responsability of receiving
many tuples from different shards in the topology and merging them.
It emmits a final result to the next bolt when it has received all
the parts or when a timeout is reached. Only results recieved so far
will be considered in the second case.
So, the emmit method from the outputCollector will be called from
two different threads.
The question is: should I synchronize the call to emmit() or it's
prepared to be thread safe?

Thanks a lot!
Darío.

Nathan Marz

unread,
Apr 9, 2012, 4:00:29 AM4/9/12
to storm...@googlegroups.com
Yes, it's thread safe.

2012/4/7 Luis Darío Simonassi <ldsim...@gmail.com>



--
Twitter: @nathanmarz
http://nathanmarz.com

Luis Darío Simonassi

unread,
Apr 9, 2012, 5:40:06 PM4/9/12
to storm-user
Great! Thanks!!!

On 9 abr, 05:00, Nathan Marz <nathan.m...@gmail.com> wrote:
> Yes, it's thread safe.
>
> 2012/4/7 Luis Darío Simonassi <ldsimona...@gmail.com>

Nathan Marz

unread,
Dec 17, 2012, 3:47:55 PM12/17/12
to storm...@googlegroups.com
That's correct, it's no longer thread safe.

On Mon, Dec 17, 2012 at 10:39 AM, Julien Letrouit <jlet...@hopper.com> wrote:
Is this still true in 0.8.0+? The following project seems to imply it is not the case:

Storm 0.8.0 emits are no longer thread safe. You may see NullPointerExceptions with DisruptorQueue in the stack trace. If you are doing emits from multiple threads or actors, you will need to synchronize your emits or have them come from a single thread.


  Julien

Julien Letrouit

unread,
Dec 18, 2012, 8:47:32 AM12/18/12
to storm...@googlegroups.com
So if one would need to perform NIO, it would involve synchronizing the output collector manually at time of tuple emission, is it correct?
It may be useful to make the following statement in the documentation tutorial explicit about this issue:

"Tuples can be emitted at anytime from the bolt -- in the prepare, execute, or cleanup methods, or even asynchronously in another thread"

  Julien

Sotaro Kimura

unread,
Dec 20, 2012, 5:24:03 PM12/20/12
to storm...@googlegroups.com
Hi. 

Excuse me for cutting in.

In DRPCTopology, 
[outputCollector emits are no longer thread safe] is not applicable. OK?

Because, CoordinatedBolt synchronizes itself.
# I tested synchronized code in own Bolt, storm deadlocked...

regards/

2012年12月18日火曜日 5時47分55秒 UTC+9 nathanmarz:
Reply all
Reply to author
Forward
0 new messages