query regarding Configuration values permitted for bolt

[Richards Peter]

Hi,

I was going through this page:
https://github.com/nathanmarz/storm/wiki/Configuration

I could see four parameters which could be overridden per spout/bolt
basis. Among them I suppose "topology.max.spout.pending" is applicable
only for spouts.

If the answer is "yes", can you tell me whether it is possible to have
a similar parameter for bolts also which could prevent it from
overflowing the queue(zeromq) or buffer(maintained in the target bolt
code by the programmer) due to slow rate of processing the tuple in
the target bolt. I am aware about the message passing system of storm
with zeromq. But I guess my second case where we use buffer is
significant.

Thanks,
Richards Peter.

[Richards Peter]

Hi,

My mistake: I am not aware about the message passing system of storm
with zeromq.

Richards Peter.

[Nathan Marz]

Yes, topology.max.spout.pending is overridable per spout, and yes, having that kind of backpressure feature would be nice. You should open up an issue for this on GitHub.

--
Twitter: @nathanmarz
http://nathanmarz.com

[Richards Peter]

Thanks Nathan,

That will be a great feature.But I don't have privileges to create issues.

Richards Peter.

[Richards Peter]

Hi Nathan,

I have one more suggestion about this feature. Can you make this value available to all the components in the topology?

That is if the topology looks like this:

S1 -> B1 -> B2 -> B3

B2 should be able to figure out how many unacked tuples can be captured from B1 and B3 should be able to figure out the same figure from B2.

Thanks,
Richards Peter.

[Nathan Marz]

Sorry... what value?

[Richards Peter]

The parameter similar to "topology.max.spout.pending"(as in Spout) which is planned for bolts.

Thanks,
Richards Peter.