I just released version 0.1.9 tonight. Overly observant Gracklers may
ask, "Hey where the hell was 0.1.8?" Well as it turns out I actually
released it earlier this month and forgot to post here about it. You
can check the changelog (http://github.com/hayesdavis/grackle/blob/
master/CHANGELOG.rdoc) for info on what was in both releases.
The 0.1.8 release adds support for specifying proxies for
Grackle::Transport. This is pretty handy if you want to use multiple
machines in a cluster with whitelisted IPs to allow you to make
enormous numbers of requests transparently or if you need to route
requests from non-whitelisted machines through a whitelisted machine.
I'm using it in production right now to pick from a set of whitelisted
machines running (the excellent and simple) tinyproxy. Here's a pretty
naive snippet for randomly choosing from a pool of proxies on each
Grackle::Client request:
#proxies is an array like ['host1:port','host2:port']
client.transport.proxy = Proc.new do |tran|
host, port = proxies.rand.split(/:/)
Net::HTTP.Proxy(host,port)
end
Obviously more intelligent proxy selection implementations could
maintain state information about the pool and load balance proxies
based on the number of requests remaining.
Have fun.
Hayes