beanstalkd work loop

140 views
Skip to first unread message

Ingo Krabbe

unread,
Jan 30, 2019, 6:01:02 PM1/30/19
to beanstalk-talk
Hi,

I detected a quite strange behaviour in my beanstalkd (go-beanstalk)-Client.

To test the behaviour of beanstalkd, you can see from my log here:

2019/01/30 13:52:55.887937 received job 51 on queue 0xc4202ed9f0
2019/01/30 13:52:55.888030 working on "hello"
2019/01/30 13:52:55.888110 delete 51
2019/01/30 13:52:55.888314 deleted 50
2019/01/30 13:52:55.941661 GET /io/send/queue HTTP/1.0 200 "OK" 0.166774
2019/01/30 13:52:56.888635 deleted 51

The /io/send/queue fires a "hello" string to the default tube.

The worker function uses Reserve(1 * time.Second) to read from the tube.

When it finds a message it pushes the job to a go channel that is worked on internally.
As a default I call a logjob.Work() function and a donejob.Work() function, which are 

ctrl.Logger.Printf("working on \"%s\"", string(j.body))

and a

go func(jid uint64) { ctrl.Logger.Printf("delete %d", jid); ctrl.Conn.Delete(jid); ctrl.Logger.Printf("deleted %d", jid) }(j.id)

I chose this operatio for delete in the background as you can see, the last delete blocks for quite exact one second before
it finishes. I can see this behaviour with the aurora console too.

When I push a new job into the queue the delete finishes and the next delete waits again. It's always only the last
job that stays in reserved mode, for the timeout the tube has been reserved.

This behaviour is a bit strange, though I can work with it, but if I would have registered another worker work flow, for example:

   Copy Job Data, Delete Job, Work on Data

The last "Delete Job" would block the "Work on Data" for the last round until it is finished, if I didn't put the delete work into background as I did here,
after I detected that behaviour.

So this behaviour is at least worth to be documented. Possibly I simply missed that fact, maybe it is a bug that noone detected yet...

Best Regards,

Ingo

Reply all
Reply to author
Forward
0 new messages