Routeing Jobs

[Jason Judge]

This isn't something I need, but am just curious on whether it is something anyone has done.

Supposing there are a bunch of beanstalkds running on a bunch of servers, for load balancing. For resilience, each job would only be submitted into a tube on localhost, bit that job could run on other servers, depending on the job type, server load etc.

Is there any kind of general routing tool or framework for beanstalkd? I'm thinking something that will be able to monitor multiple tubes on multiple servers, perhaps do a little job data inspection, and then move those jobs to other tubes on other servers as needed?

Actually, I kind of do need something like this. We have a shop on one server, and I want to push the order and payment details into a local tube (for speed and resilience). On another server there is an application that pushes these details to a CRM. Now, that application could read the source pipe on the shop server (and that may still be the best way to do it), but I suspect it would be better to move those source jobs from the shop local pipes to the CRM server local pipes, then those processes also only need to handle local pipes, and don't have to worry about networks being up or down.

So in my case I would just want to take all jobs from pipe A on server B, and move them to pip C on server D. No data inspection needed - just a plain and simple move, so a very simple example of routeing. Are there tools to do that available? Or is this something I would need to write. I don't want to reinvent the wheel, so would prefer to use what is already out there.

I deal mainly with PHP, shell scripting, and run the Python tools (but don't code Python), and don't touch Ruby, in case that helps.

-- Jason