I'm hoping someone can clarify something for me regarding rate limiting and Phirehose.
I'm trying to use Ghetto Collector and have had gotten it working. I'm running an Amazon virtual machine with Ubuntu and spawn a new process for each item I want to track.
When I have 1, it works flawlessly.
When I have 2, it works flawlessly.
When I have 3, I start getting (but not always) the Rate Limited (RL) message "Enhance Your Calm".
When I have 4, most of the time I get the RL message.
When I have 5, I continuously get the RL message.
Now even when I have 5, it seems the most active streams stick around but the other three drop out due to the "PhirehoseConnectLimitExceeded" error (going over 20 retries). What is it "trying" to do that gets rate limited?
I thought that once the stream was open, there were no more real "requests". It just caught whatever was being thrown at it.
I'm really trying to play by the rules but I think I have a basic misunderstanding of how Phirehose and/or the Firehose stream work.
Any help will be greatly appreciated. Phenomenal help will be rewarded with cookies.
Scott.