Job queue drops after GRAB_JOB command

[Konstantin Timerkhanov]

Hi there!

Sorry if I'm asking a stupid question, but I just can't figure it out or find any information about it.

I have a queue with plenty of jobs (let's call it "Foo") and Gearman (version 1.1.18+ds) with in-memory storage. I tried using node.js with node-gearman and abraxas clients, and the behavior is the same.

After restarting my application, if there are no additional listeners for the queue "Foo" (i.e., the count of workers is 0, and let's assume the number of jobs is 100,500), then when a new worker subscribes, all the jobs that were in the queue are dropped. After debugging, I found out that the queue drops after the GRAB_JOB command. Before this, and after CAN_DO, I can see by status that there are still jobs in the queue and the worker is connected.

Can somebody please explain why? Is there any way to keep jobs per application restart?

[Edward J. Sabol]

Are you stopping gearmand and restarting the gearmand process? That's the only way you're going to lose jobs that have been queued up.

Also, you seem to be doing things in an unusual order. You should start gearmand first and leave it running. You should then start your workers which connect to gearmand. Keep your workers running. You shouldn't have zero workers ever, really. Then your clients submit jobs to gearmand with work payloads and gearmand returns the results.

Your gearmand is version 1.1.18, which is several years old at this point. Although I'm not aware of any known bug that would result in what you describe, I still recommend upgrading to the latest version even if you have to compile it yourself. Docker images are available. I like this one: https://github.com/artefactual-labs/docker-gearmand

If you need further assistance, GitHub is the best place to submit an issue. https://github.com/gearman/gearmand/

Regards,

Ed

[Clint Byrum]

As described, that would be a very serious bug.

But we need some more information.

Are these submitted as background jobs? Foreground jobs are dropped if and when their client disconnects. Maybe that's what's going on? 

Anyway let us know what kind of jobs they are. If you can give us dumps of the status commands, and even better, debug level logging from the server when this happens, we can better understand what's going on.

--
You received this message because you are subscribed to the Google Groups "Gearman" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gearman+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gearman/d5183adc-01b1-47a9-950d-f36a8ce97648n%40googlegroups.com.

[Konstantin Timerkhanov]

Oh my gosh.

Thanks you a lot! I did submit foreground jobs instead of background.

(Still  found strange that there is no highlighted info about such default behavior with common submitJob methods in node.js modules, but this is not important right now)