Best approach for monitoring gearman queue processing status

1,089 views
Skip to first unread message

Vivek

unread,
Oct 20, 2013, 11:30:24 AM10/20/13
to gea...@googlegroups.com
Hi, 
I want to monitor if jobs are getting picked up from a particular queue or not. 
Currently I was doing this by firing  ./gearadmin --status and checking the diff. If diff is non zero then there is something failed. 
If something failed I will repeat this process 4 times with 5 minute delay. If all returned failed I was generating alert. 

This was working fine when load was low but this approach will not work with high load. As there can be chances that there is newborn job which was not picked by some worker. 

Is there admin command to find out last picked up job time and next pending job creation time?
or 
 Is there any better way to monitor the queue status ?

Jeff Minard

unread,
Oct 20, 2013, 11:26:16 PM10/20/13
to gea...@googlegroups.com
If you were to swap over to a persistent stored backend, like mysql/maira, doing a quick db check on the queue table would be a rather painless way of checking job in/out amounts.

Also, it sounds like you're working under the assumption that all jobs in a given queue must be farmed out at any given time. I suppose if that's your goal, that's fine, but typically people using gearman allow for queues to burst up in size for short periods of time. It's one of the nice things about the queue is that you can fire off lots of jobs and not worry (too much) about bursts -- they jobs will get done, but perhaps not immediately.

Finally:

Is there admin command to find out last picked up job time and next pending job creation time?

How would gearmand know when your clients are next going to queue a job? If you mean, "a command to find out when gearmand will next assign out a queued command" it can't really know that either because it depends entirely on how long your workers take to free up.
Reply all
Reply to author
Forward
0 new messages