peek job

[esha...@gmail.com]

Is there a way in the async observer API I can find the current status
of a job?

If I declare a queue like so:
AsyncObserver::Queue.queue = Beanstalk::Pool.new(%w(localhost:
11300))

I can retrieve general stats:
AsyncObserver::Queue.queue.stats

and I can find out the last job to be addressed:
AsyncObserver::Queue.queue.peek


But I don't see a way to find out the status of a particular job.

[esha...@gmail.com]

Defining peek_job(id) in connection.rb as below kind of does what I
want. It lets me know that a job is still in process. But it returns
nil both for completed jobs as well as jobs that never existed. Also,
I found that it would periodically return an UnexpectedResponse, which
seemed to be neither here nor there; probably something related to
open_connections. Any ideas?

class Pool
...
def peek_job(id)
open_connections.each do |c|
begin
job = c.peek_job id
return job if job
rescue UnexpectedResponse => e
end
end
nil
end

[Keith Rarick]

On Thu, Apr 10, 2008 at 12:18 AM, esha...@gmail.com <esha...@gmail.com> wrote:
> Defining peek_job(id) in connection.rb as below kind of does what I
> want.

Sorry, I didn't mean to omit peek_job in the Pool interface. I'll send
out beanstalk-client 0.10.0 later today; it'll have peek_job,
peek_ready, peek_delayed, and peek_buried.

I'm not sure why you get UnexpectedResponse -- is it possible you're
running mismatched versions of beanstalkd and beanstalk-client? What
is the response (i.e. what is the output of puts e.inspect in your
implementation of peek_job)?

kr

[esha...@gmail.com]

Could be a mismatched version. I'm running the beanstalkd 0.10 and
beanstalk-client-0.9.0. I notice that RawConnection.peek swallows the
UnexpectedResponse exception whereas RawConnection.peek_job does not.
Any reason for the inconsistency?

def peek()
interact("peek\r\n", :job)
rescue UnexpectedResponse
nil
end

def peek_job(id)
interact("peek #{id}\r\n", :job)
end

I look forward to seeing the client 0.10.0!

Thanks for the assistance,
Eric

On Apr 10, 12:51 pm, "Keith Rarick" <k...@causes.com> wrote:

[Keith Rarick]

On Thu, Apr 10, 2008 at 2:11 PM, esha...@gmail.com <esha...@gmail.com> wrote:
> Could be a mismatched version. I'm running the beanstalkd 0.10 and
> beanstalk-client-0.9.0.

Then I would expect peek to fail and peek_job to work for you, unless
the job doesn't exist (see below).

> I notice that RawConnection.peek swallows the
> UnexpectedResponse exception whereas RawConnection.peek_job does not.
> Any reason for the inconsistency?

No, in fact that's why you were seeing UnexpectedResponse. It's fixed it in git.

kr

[esha...@gmail.com]

This seems to work OK:
def peek_job(id)
@socket.write("peek #{id}\r\n")
begin
Job.new(@jptr, *read_job('FOUND'))
rescue UnexpectedResponse
nil
end
end

I notice that if a job with this ID has not yet been submitted OR if
the job has already concluded processing successfully I get nil back.
Only if the job is still queued do I get a response. Successful jobs
are deleted, per the beanstalkd

So given that constraint, my strategy will be to check the total-jobs
variable:
AsyncObserver::Queue.queue.stats['total-jobs']

And if my job ID is less than or equal to total-jobs I can rest
assured the job has completed. If the total-jobs is less than my job
ID, a restart on beanstalkd likely occurred. I could also keep track
of the job ID and the time it was submitted, then compare that to the
uptime in stats.

Even better we could change the API so that when a job is submitted it
returns not just the ID but current unixtime. Then when you call
peek_job, the method can determine based on stats whether the job has
disappeared.

I noticed that there is a job_stats method on RawConnection, If you're
adding to the Pool API, you might as well add that to Pool.

Thanks for a great innovative plugin!

--> Eric

[Keith Rarick]

On Thu, Apr 10, 2008 at 3:07 PM, esha...@gmail.com <esha...@gmail.com> wrote:
> I notice that if a job with this ID has not yet been submitted OR if
> the job has already concluded processing successfully I get nil back.
> Only if the job is still queued do I get a response. Successful jobs
> are deleted, per the beanstalkd

Yes.

> So given that constraint, my strategy will be to check the total-jobs
> variable:
> AsyncObserver::Queue.queue.stats['total-jobs']

> ...

That's reasonable.

> Even better we could change the API so that when a job is submitted it
> returns not just the ID but current unixtime. Then when you call
> peek_job, the method can determine based on stats whether the job has
> disappeared.
>
> I noticed that there is a job_stats method on RawConnection, If you're
> adding to the Pool API, you might as well add that to Pool.

This is exposed as Job#stats. So conn.peek_job(123).stats would work
(though it would result in an extra round trip).

> Thanks for a great innovative plugin!

You're most welcome! I hope you find it useful.

kr