Possible modifications to reserve-with-timeout
Kaspar Schiess
TLDR: I would like to ask if beanstalkd might be amended to include a
feature that allows select(2) calls on sockets connected to it. If there
is interest, I would be willing to implement and propose patches.
Here's some background on what I need and why I need it:
I am experimenting/implementing a cross process communications framework
in Ruby called 'cod' [1]. The goal of this framework is to allow a
no-brainer approach to IPC.
A lot of that work involves doing things the 'old' unixy way, using
processes not threads/callbacks. So I need to be able to rely on
select(2) calls that block processes until something happens.
The current beanstalkd protocol allows me to issue reserve-with-timeout
<seconds>, but that only gets me halfway there. Two issues with that
call remain: [2]
1) Only to be able to specify 1 second min. is really too long a delay.
I would like to be able to specify at least tenths of a second, if not a
finer granularity.
2) If I select on beanstalk-connected sockets and normal tcp sockets
simultaneously, one of the normal sockets might be ready before the
timeout of reserve-with-timeout expires. The beanstalk connection then
stays locked until that timeout passes.
Implementation: I would probably explore along the lines of an 'abort'
command, that aborts a reserve-with-timeout prematurely. Not sure if
this matches the way beanstalkd is implemented, but I guess I would find
out quickly.
Interested? Welcome addition? Bad idea?
best regards,
kaspar
[1] https://github.com/kschiess/cod
[2]
https://github.com/kschiess/cod/blob/master/spec/acceptance/beanstalk_spec.rb#L121
Keith Rarick
> 1) Only to be able to specify 1 second min. is really too long a
> delay. I would like to be able to specify at least tenths of a second, if
> not a finer granularity.
That's a reasonable request. Could you file an issue?
> 2) If I select on beanstalk-connected sockets and normal tcp sockets
> simultaneously, one of the normal sockets might be ready before the timeout
> of reserve-with-timeout expires. The beanstalk connection then stays locked
> until that timeout passes.
That's how sockets are supposed to work. The select call will tell
you which fds are ready. If the beanstalkd connection isn't ready,
don't read from it.
> Implementation: I would probably explore along the lines of an 'abort'
> command, that aborts a reserve-with-timeout prematurely. Not sure if this
> matches the way beanstalkd is implemented, but I guess I would find out
> quickly.
This is more interesting, and hints at the real problem I suspect
you're asking about:
If the client is currently waiting for a reserve response and has
other commands it wants to send, it can't send them immediately. It
either has to wait or open another connection.
This has come up several times before, and I agree it's a problem.
Let's see what it would take to fix.
It would actually be quite easy to have beanstalkd abort a waiting
reserve command when *any* other command appears on the connection,
but I'm afraid that behavior might be confusing. Also, there's no
good way to indicate this case in the response of the existing
reserve command. (This is a design flaw in the v1.0 beanstalkd
protocol. It should have allowed for the possibility of introducing
new error codes in the future.) We could work around both of these
problems by making a new command that's like reserve, but can be
aborted.
Would that serve your needs?
kr
Kaspar Schiess
Thanks for the detailed answer and for interpreting my babble right. Of
course I mean that I want to issue more commands on the line that is
still reserved because my select has fired on other sockets. I just
didn't make that very clear.
A new command like reserve-with-timeout, but allowing an abort? Yes that
would serve my purpose quite nicely. I have tried to think of a good
name for it, but I guess I am stuck in my libraries logic there,
'select' is all I can come up with.
How would allowing an 'abort' command sent after the old
'reserve-with-timeout' break existing clients?
kaspar
Keith Rarick
> How would allowing an 'abort' command sent after the old
> 'reserve-with-timeout' break existing clients?
It wouldn't, but the implementation would be trickier,
and I think it would present an interface with more
potential for confusion.
kr