MIME-Version: 1.0
Message-ID: <263bd9bd-e0ae-4b72-80b8-6e2223f1a1f6@r66g2000hsg.googlegroups.com>
Date: Mon, 12 May 2008 06:45:37 -0700 (PDT)
Received: by 10.150.50.17 with SMTP id x17mr155997ybx.28.1210599937199; Mon, 
	12 May 2008 06:45:37 -0700 (PDT)
In-Reply-To: <e6a042fe-1d8d-4c80-8188-1aa89a70ea2f@z72g2000hsb.googlegroups.com>
X-IP: 98.212.155.215
References: <c4ee8498-af4f-4b3c-8d3a-82916b681e20@f36g2000hsa.googlegroups.com> 
	<e6a042fe-1d8d-4c80-8188-1aa89a70ea2f@z72g2000hsb.googlegroups.com>
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.12) 
	Gecko/20080129 Firefox/2.0.0.5 (Debian-2.0.0.12-1),gzip(gfe),gzip(gfe)
Subject: Re: Deadline soon explained
From: Erich <sophac...@gmail.com>
To: beanstalk-talk <beanstalk-talk@googlegroups.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Good question. This is something I definately should have been clearer
about.The case you describe below will not starve workers here is why:

beanstalkd will only return a DEADLINE_SOON when there is a job about
to expire AND there are no jobs to send to the requesting connection.
That is, DS prevents blocking, so if there is a job on the queue, it
will be sent immediately.

Regards,
Erich

igrigorik wrote:
> Erich, thanks for the detailed explanation.. A few questions coming
> out of it:
>
> Let's assume you have a 'Manager' process, whose role is to talk to
> beanstalk and reserve/release jobs when required. As well, there are
> several workers hidden behind the Manager. The data flow is as
> follows: Manager takes a job off the queue; Manager passes a job to an
> empty worker; Worker begins processing; Manager is waiting for the
> next worker to finish, in order to take off the next job.
>
> Now, given the scenario you've described. Manager will only have one
> connection open to beanstalk, correct?
>
> Question: if a manager has 3 workers, all doing some computations on
> separate jobs from Beanstalkd, and if one of the workers get close to
> their TTR, next time Manager tries to reserve a job (possibly for
> another, non-slow worker), Beanstalkd will return a DEADLINE_SOON
> error. Is that correct?
>
> I can see why it's implemented the way it is, but how do you recommend
> working around the scenario described above? One long-running worker
> close a deadline, essentially forces all other workers to starve for
> work...
>