Gearman dropping job with unique key

[Vivek]

HI,
I am using gearman unique key feature to avoid duplication of the job.
As Gearman doesn't support requeue task for a job I have implemented requeue like this:

1. Gearman Worker sends job completed notification to the server using gearman_job_send_complete.(It returns GEARMAN_SUCCESS I am not getting GEARMAN_IO_WAIT)
   
2. Gearman Worker finalize the the job using gearman_job_free.
3. Gearman client add a new job with same unique key. (It returns success)
   

What problem I am facing:

  Gearman is ignoring the new job in high load.

Worker options I am using are:

1. GEARMAN_WORKER_GRAB_UNIQ
2. GEARMAN_WORKER_NON_BLOCKING

What could be the problem here ?

[Brian Aker]

Hi,

On Dec 5, 2012, at 6:53 AM, Vivek <goelvi...@gmail.com> wrote:

> As Gearman doesn't support requeue task for a job

Do you mean requeue on error?

A worker can say that it was unable to complete a job, and the job will go back onto the queue.

Cheers,
-Brian

[Vivek Goel]

No I have circular job. So I after completing the job I am submitting the same job again using same process. 

regards

Vivek Goel

[Vivek]

> As Gearman doesn't support requeue task for a job

Do you mean requeue on error?

A worker can say that it was unable to complete a job, and the job will go back onto the queue.

Hi, Brian
Which binary packet will serve this purpose ? I tried to use packet WORK_FAIL,  but using that packet drops job from server. 

 

Cheers,
        -Brian

[Vivek Goel]

Ok got . There is no function to tell the server that worker is not able to process job now.
http://www.pal-blog.de/entwicklung/perl/retry-failed-gearman-jobs.html

--
You received this message because you are subscribed to a topic in the Google Groups "Gearman" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gearman/5qISg9rwStw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gearman+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Brian Aker]

Hi!

On Sep 22, 2013, at 11:32 AM, Vivek Goel <goelvi...@gmail.com> wrote:

Ok got . There is no function to tell the server that worker is not able to process job now. 
http://www.pal-blog.de/entwicklung/perl/retry-failed-gearman-jobs.html

I will need to go read the blog but if it asserting what I think it is asserting it is sorta wrong.

The way the gearman server works is that any worker that drops a connection forces the server to retry a job.

How does this work in implementation? In libgearman if you return GEARMAN_FATAL or call an exception the server will cancel the job and send any client the appropriate response (assuming the client is synchronous).

If you return GEARMAN_ERROR? Then libgearman bounces the connection which causes the server to retry the job (and job retry can be manipulated via the admin protocol).

Does that help clear this up?

Cheers,

-Brian

[Vivek]

If I understood correctly throwing this error will cause dropping the connection. If my worker's is having 1 -> n relation (one worker processing more than n jobs)  this will result in dropping all n jobs. 

But I want to requeue only one job not all.  

[Brian Aker]

Hi!

On Oct 10, 2013, at 12:26 PM, Vivek <goelvi...@gmail.com> wrote:

If I understood correctly throwing this error will cause dropping the connection. If my worker's is having 1 -> n relation (one worker processing more than n jobs)  this will result in dropping all n jobs. 

But I want to requeue only one job not all.  

Something like this is needed:

GEARMAN_COMMAND_WORK_RETRY,           /* W->J/C: HANDLE [0]PAUSE[0]WARN_TEXT */

Handle -> Job to retry

Pause ->  Amount of time before retry

Warning Text -> A message to be sent back to the client to tell it that a retry happened.

Cheers,

-Brian