We've noticed high CPU in an idle pybeanstalk process which maintains 20 ServerConns (one for each of 20 threads). Upon investigating, we found that ServerConn._get_response() has a poll() with a 1 ms timeout! So we're getting 20,000 timeouts per second (and probably bashing the heck out of the GIL).
We'll probably end up redesigning the application to only have a single beanstalk connection and distribute jobs internally to the various threads. That at least will reduce the spinning to 1000/sec, but that still seems crazy. Why is there such a short timeout?
---
Roy Smith
r...@panix.com