Question about Design Consideration in AbstractSpout (regarding es.status.ttl.purgatory)

[dennis...@googlemail.com]

Hello there,

I was wondering, if somebody may be able to help me understand the decision in having the InProcessMap "beingProcessed" in the AbstractSpout with its own timeout  es.status.ttl.purgatory.

I imagine this was due to a case where neither the successful or unsucessful processing of a URL led to a fail() or ack() AND also the topology.message.timeout.secs did not clean up a tuple that timed out somewhere in the topology. What is the case where you would want this es.status.ttl.purgatory to hit?

Thanks a lot,

Dennis

[DigitalPebble]

Hi Dennis

See http://digitalpebble.blogspot.com/2016/10/whats-new-in-stormcrawler-12.html for the explanation

Thanks

Julien

--
You received this message because you are subscribed to the Google Groups "DigitalPebble" group.
To unsubscribe from this group and stop receiving emails from it, send an email to digitalpebbl...@googlegroups.com.
To post to this group, send email to digita...@googlegroups.com.
Visit this group at https://groups.google.com/group/digitalpebble.
For more options, visit https://groups.google.com/d/optout.

--

 
Open Source Solutions for Text Engineering
 
http://www.digitalpebble.com

http://digitalpebble.blogspot.com

@digitalpebble

[dennis...@googlemail.com]

Hi Julien,

thanks a lot! I misread the code of the InProcessMap implementation. I did not get that it keeps the url for es.status.ttl.purgatory seconds after remove is called. I thought it was keeps it until remove is called OR es.status.ttl.purgatory, which did not seem to make sense. Now it does exactly what I thought it should do!

Thanks for getting me back on track.

I really like the project and your responsiveness! Thanks a lot for that!

Dennis

[DigitalPebble]

Hi Dennis

You are welcome. Feel free to give any feedback and / or tell us what you are using SC for.

Julien