Socket issues with pika and rabbitmq

[Ender Savage]

Hey guys,

I've been battling a bug in my application for a while now and was hoping to get some insight.  

Ultimately i am getting the error code Socket Error on fd 524: 10054.  I have a consumer that does a job that takes anywhere between 10 seconds to 30 minutes.  If it does a job that takes 20+ minutes, my consumer seems to exit despite adding in some reconnection and disconnect handlers.  attached is a screen shot of some debugging when this occurs.  Any help would be greatly appreciated.

[Gavin M. Roy]

So from the log output, it looks like you’re doing sftp inline in the consumer app?

My guess is you’re blocking the IOLoop and process with that and thus RabbitMQ is disconnecting you after not responding to the 5 minute heartbeat ping.

Gavin

--
You received this message because you are subscribed to the Google Groups "Pika" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pika-python...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
<Screen Shot 2014-08-19 at 9.56.04 PM.png>

[Ender Savage]

I am indeed doing some sftp within the consumer app.  Do you have any suggestions to avoid blocking the IOLoop?

Perhaps its better i do the transfer outside of the consumer loop?  but after i transfer the file and send a message to the client queue i need the IOLoop to continue with more jobs

Thanks for the quick response

[Gavin M. Roy]

You could start a thread for transferring the file and use IOLoop.add_timeout functionality to check if the thread is finished once a second or so.

https://docs.python.org/2/library/threading.html

http://pika.readthedocs.org/en/0.9.13/modules/adapters/select.html#pika.adapters.select_connection.SelectConnection.add_timeout

[Ender Savage]

I dont think its the transfer that's taking more than 5 minutes.  There's another part of the consumer that is performing a function on a list of files.  That portion can take well over 5 minutes to complete.  So i think i should try spawning a thread for the function that takes 5+ minutes to complete as to not interfere with the 5 minute heart beat interval and get disconnected by rabbitmq?

thanks again!

[Gavin M. Roy]

Yeah that makes sense to me.