best practice for cleaning binlog folder?

chadkouse

unread,

Jan 21, 2011, 1:10:38 PM1/21/11

to beanstalk-talk

I had to stop beanstalkd today for a minute, and when I restarted it I
waited like 5 minutes and it still hadn't come back up. Turned out I
had over 15000 binlog files and this was causing the server to take a
long time to start.

I'm sure most of these contain jobs that were processed and deleted
long ago.

What is the best practice for pruning unneeded binlog files?

jab_doa

unread,

Jan 22, 2011, 9:41:33 AM1/22/11

to beanstalk-talk

Hi chadkouse,

do you have buried jobs in your queue? They will prevent your binlog
from getting deleted. This is a known limitation, but afaik kr is
working on it.

Regards,
Jan

Chad Kouse

unread,

Jan 22, 2011, 9:54:55 AM1/22/11

to beansta...@googlegroups.com

No we don't have any buried jobs.

--chad

> --
> You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
> To post to this group, send email to beansta...@googlegroups.com.
> To unsubscribe from this group, send email to beanstalk-tal...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
>

jab_doa

unread,

Jan 22, 2011, 9:58:43 AM1/22/11

to beanstalk-talk

What does cmd-list-tubes return? Run cmd-stats-tube on each of them.
How many jobs are in there?

Jan

Chad Kouse

unread,

Jan 22, 2011, 10:03:06 AM1/22/11

to beansta...@googlegroups.com

Ah good thinking I will see if any orphaned jobs are in tubes from testing / development and report back

--chad

Peter Portante

unread,

Jan 22, 2011, 9:40:24 AM1/22/11

to beansta...@googlegroups.com

We have found that if there is one job that is left stranded in the system, all the binlogs are not removed until that one stranded job is removed.

We have a bug in our code where we don't place a timeout on a job, so for now we resort to periodically deleting that stranded to job to have beanstalkd automatically clean up the bin logs.