Multiple sprouts linking back to a identical third one

[Lionel Touati]

Hi List,

I've got a question wrt storm. I'd like to create the following topology ( not even sure its possible).

S1-----

         |____S3

         |

S2-----

Both S1 & S2 emit a tupple for consuption by S3

Here's a sample of my topology

        builder.setBolt("ipDetectionBolt", new IPAddressDetectorBolt()).shuffleGrouping("retrieveMessageFromQueueSpout");

        builder.setBolt("pageDetectionBolt", new PageDetectorBolt()).shuffleGrouping("retrieveMessageFromQueueSpout");

        builder.setBolt("customerDetectionBolt", new CustomerDetectionBolt()).shuffleGrouping("ipDetectionBolt").shuffleGrouping("pageDetectionBolt");

The problem is that instead of submitting both tupples to S3, S1 get executed with the tupple of S1 and then with the tupple of S2.

How can I sync both ?

Thanks a lot for your help

[abhinav]

Lionel,

Could you clarify a bit further with some additional details about S1, S2 and S3? Are they spouts or bolts?

What do you mean by "sync both"?

Thanks,

--Abhi

[Lionel Touati]

Hi Abhi,

Sorry for the confusion, yes both S1 and S2 are BOLTS, the idea is
that when S3 is called it has results generated by S1 and S2

Thanx for your help

On Mar 16, 6:57 am, abhinav <abhinav...@gmail.com> wrote:
> Lionel,
>
> Could you clarify a bit further with some additional details about S1, S2
> and S3? Are they spouts or bolts?
> What do you mean by "sync both"?
>
> Thanks,
> --Abhi
>
>
>
>
>
>
>
> On Thu, Mar 15, 2012 at 2:31 PM, Lionel Touati <ltou...@gmail.com> wrote:
> > Hi List,
>
> > I've got a question wrt storm. I'd like to create the following topology (
> > not even sure its possible).
>
> > S1-----
> >          |____S3
> >          |
> > S2-----
>
> > Both S1 & S2 emit a tupple for consuption by S3
>
> > Here's a sample of my topology
>
> >         builder.setBolt("ipDetectionBolt", new
> > IPAddressDetectorBolt()).shuffleGrouping("retrieveMessageFromQueueSpout");
> >         builder.setBolt("pageDetectionBolt", new
> > PageDetectorBolt()).shuffleGrouping("retrieveMessageFromQueueSpout");
> >         builder.setBolt("customerDetectionBolt", new

> > CustomerDetectionBolt()).shuffleGrouping("ipDetectionBolt").shuffleGrouping ("pageDetectionBolt");

[abhinav]

Lionel,

You can use a simple synchronization method like this: https://gist.github.com/2049902

You'll need to emit the messageId of the message you fetch from the QueueSpout in bolts S1 and S2.

If you want to increase the parallelism on your bolts, you should use a FieldGrouping on the messageId field for S3.

Cheers,

Abhi

[Lionel Touati]

Hi Abhi,

Thanks a lot, it works great