Message from discussion Deadline soon explained
Date: Mon, 12 May 2008 06:45:37 -0700 (PDT)
Received: by 10.150.50.17 with SMTP id x17mr155997ybx.28.1210599937199; Mon,
12 May 2008 06:45:37 -0700 (PDT)
X-HTTP-UserAgent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:22.214.171.124)
Gecko/20080129 Firefox/126.96.36.199 (Debian-188.8.131.52-1),gzip(gfe),gzip(gfe)
Subject: Re: Deadline soon explained
From: Erich <sophac...@gmail.com>
To: beanstalk-talk <email@example.com>
Content-Type: text/plain; charset=ISO-8859-1
Good question. This is something I definately should have been clearer
about.The case you describe below will not starve workers here is why:
beanstalkd will only return a DEADLINE_SOON when there is a job about
to expire AND there are no jobs to send to the requesting connection.
That is, DS prevents blocking, so if there is a job on the queue, it
will be sent immediately.
> Erich, thanks for the detailed explanation.. A few questions coming
> out of it:
> Let's assume you have a 'Manager' process, whose role is to talk to
> beanstalk and reserve/release jobs when required. As well, there are
> several workers hidden behind the Manager. The data flow is as
> follows: Manager takes a job off the queue; Manager passes a job to an
> empty worker; Worker begins processing; Manager is waiting for the
> next worker to finish, in order to take off the next job.
> Now, given the scenario you've described. Manager will only have one
> connection open to beanstalk, correct?
> Question: if a manager has 3 workers, all doing some computations on
> separate jobs from Beanstalkd, and if one of the workers get close to
> their TTR, next time Manager tries to reserve a job (possibly for
> another, non-slow worker), Beanstalkd will return a DEADLINE_SOON
> error. Is that correct?
> I can see why it's implemented the way it is, but how do you recommend
> working around the scenario described above? One long-running worker
> close a deadline, essentially forces all other workers to starve for