ruby client delete

[danm...@gmail.com]

I have a ruby client and I can get and print messages but I can never
call delete?

def put(msg)
beanstalk = Beanstalk::Pool.new(['localhost:11300'])
beanstalk.put(msg.to_s)
end

def take
beanstalk = Beanstalk::Pool.new(['localhost:11300'])
job = beanstalk.reserve
puts job.body
job.delete
end

if I call put once then take, the take gives this error:

`delete': undefined method `delete' for #<Beanstalk::Pool:0x1326d50>
(NoMethodError)

thoughts?

[Dustin]

On Jun 17, 2:01 pm, "danma...@gmail.com" <danma...@gmail.com> wrote:
> I have a ruby client and I can get and print messages but I can never
> call delete?
>

>   def take
>     beanstalk = Beanstalk::Pool.new(['localhost:11300'])
>     job = beanstalk.reserve
>     puts job.body
>     job.delete
>   end
>
> if I call put once then take, the take gives this error:
>
> `delete': undefined method `delete' for #<Beanstalk::Pool:0x1326d50>
> (NoMethodError)

That error message doesn't match your code there.

[danm...@gmail.com]

Part of the confusion is the weird error messag, I am calling delete
on a job, so why does it say Pool?

my beanstalk_client.rb

require 'beanstalk-client'

class BeanstalkClient

def initialize()
@beanstalk = Beanstalk::Pool.new(['localhost:11300'])
end

def put(msg)

beanstalk = Beanstalk::Pool.new(['localhost:11300'])

beanstalk.put(msg.to_s)
end

def take
beanstalk = Beanstalk::Pool.new(['localhost:11300'])
job = beanstalk.reserve
puts job.body
job.delete
end

bean_client = BeanstalkClient.new
bean_client.put("test msg")
bean_client.take
end

dmayer$ ruby ./lib/queues/beanstalk_client.rb
connecting to beanstalk at localhost:11300
connecting to beanstalk at localhost:11300
connecting to beanstalk at localhost:11300
test msg
/opt/local/lib/ruby/gems/1.8/gems/beanstalk-client-1.0.0/lib/beanstalk-
client/job.rb:47:in `delete': undefined method `delete' for
#<Beanstalk::Pool:0x100a108> (NoMethodError)
from ./lib/queues/beanstalk_client.rb:18:in `take'
from ./lib/queues/beanstalk_client.rb:23

[danm...@gmail.com]

I got this to work, but only using beanstalk-client-0.11.0.tgz, if I
use the gem or the beanstalk-client-1.0.0.tgz I get the same error as
reported above.

Trying to simplify more I made testing.rb which works and does the
delete on the 0.11 release mentioned above

require 'lib/beanstalk-client.rb'

def put_msg(msg)

beanstalk = Beanstalk::Pool.new(['localhost:11300'])
beanstalk.put(msg.to_s)
end

def take_msg
#beanstalk = Beanstalk::Pool.new(['localhost:11300'])

beanstalk = Beanstalk::Pool.new(['localhost:11300'])
job = beanstalk.reserve
puts job.body
job.delete
end

put_msg("test msg")
take_msg

So am I doing something a old way or missing something about how 1.0
works?

I guess for now I will just start building on 0.11 , but I would be
interested in knowing what is up... if anyone has ideas.

[danm...@gmail.com]

More to share, I just found on another thread KR saying this is fixed
just not in the gem or 1.0 tar but in the github tar

I downloaded
http://github.com/kr/beanstalk-client-ruby/tarball/9faa74

and using that it works as well... So I guess I am on 1.0.x? and
things are working great, switching from Amazon SQS to this has been a
lifesaver in terms of performance.

On Tue, Jun 3, 2008 at 3:02 PM, Frédéric Logier <fre...@gmail.com>
wrote:
> Hi, I have some problems with this version and beanstalkd (0.11 and
> 1.0) :

This is fixed in the git repo, but not released yet. Try the tarball
at http://github.com/kr/beanstalk-client-ruby/tarball/9faa74 to get it
working right away.

kr

[Keith Rarick]

On Tue, Jun 17, 2008 at 11:40 PM, danm...@gmail.com <danm...@gmail.com> wrote:
> More to share, I just found on another thread KR saying this is fixed
> just not in the gem or 1.0 tar but in the github tar

Sorry about that! I just put version 1.0.2 on rubyforge.

> I downloaded
> http://github.com/kr/beanstalk-client-ruby/tarball/9faa74
>
> and using that it works as well...

I'm glad it's working now.

> So I guess I am on 1.0.x? and
> things are working great, switching from Amazon SQS to this has been a
> lifesaver in terms of performance.

That's great news. Do you have any numbers to share? I'm always
curious about how people are using beanstalk.

kr

[danm...@gmail.com]

Yeah, I have some performance numbers, we haven't cleaned up all the
data yet. I will be making a post about comparing Rinda, SQS,
Starling, Beanstalk, and Ruby in memory queues soon. We ended up going
with Beanstalk because it was the fastest queue system that we could
distribute over many machines (EC2), it blew SQS out of the water. I
will drop the link here after I do a write up, but I just got back
into town and I am currently playing catch up.

peace,
Dan

On Jun 18, 7:54 pm, "Keith Rarick" <k...@causes.com> wrote:

[Keith Rarick]

On Mon, Jun 23, 2008 at 8:18 PM, danm...@gmail.com <danm...@gmail.com> wrote:
> I will drop the link here after I do a write
> up, but I just got back into town and I am
> currently playing catch up.

Thanks! I look forward to it.

kr

[danm...@gmail.com]

Here is the post comparing a bunch of Ruby messaging / queueing
systems

http://devver.net/blog/2008/06/ruby-messaging-shootout/

I posted our analysis and thoughts in the post, and a bunch of
additional stats, but here is the really quick overview below.

The basic info:
10 messages, remote queue servers on a single EC2, while the client
was on machine
Queue type user system total real
MemoryQueue: 0.000000 0.000000 0.000000 ( 0.000067)
LocalStarlingQueue: 0.000000 0.010000 0.010000 ( 0.015040)
BeanstalkClient: 0.010000 0.000000 0.010000 ( 0.005700)
SQS1: 0.120000 0.050000 0.170000 ( 10.608450)
BeanstalkClientRemote: 0.020000 0.030000 0.050000 ( 4.263844)
RemoteStarlingQueue: 0.010000 0.020000 0.030000 ( 3.366750)
100 messages between two EC2 instances
BeanstalkClientRemote: 0.010000 0.000000 0.010000 ( 0.793841)
RemoteStarlingQueue: 0.010000 0.000000 0.010000 ( 1.067932)

On Jun 23, 10:11 pm, "Keith Rarick" <k...@causes.com> wrote:

[Keith Rarick]

On Mon, Jun 30, 2008 at 9:01 AM, danm...@gmail.com <danm...@gmail.com> wrote:
>
> Here is the post comparing a bunch
> of Ruby messaging / queueing systems

Interesting. I had no idea starling was so fast. Thanks for doing this work.

One question about your methodology: 100 messages seems few enough
that it would be prone to noise. Did you run multiple tests to get a
statistically significant sample?

kr

[Paul Dix]

I agree about the message count. I'd more interested in runs of 10k
messages or more.

Paul

[Dustin]

On Jun 30, 12:39 pm, "Paul Dix" <p...@pauldix.net> wrote:
> I agree about the message count. I'd more interested in runs of 10k
> messages or more.

It'd be good to turn that around a bit. How many messages can you
get in and out in a unit of time vs. how many it takes to do a
specific amount? With these numbers, I don't think there's any
significant difference between the implementations.

[danm...@gmail.com]

I actually ran quite a few tests with 1000 msg, but didn't include the
results. It seems people want an even higher message count. I think I
will set up a run this evening and let it run on 10,000 msgs before I
head off to bed and I can post the results tomorrow. I do agree 100 is
pretty low, we did run the staging runs of 100 before we started
timing things to try to remove some noise from the system.

>> It'd be good to turn that around a bit. How many messages can you
get in and out in a unit of time vs. how many it takes to do a
specific amount? With these numbers, I don't think there's any
significant difference between the implementations.

As for this doesn't the mean put and take show what the difference on
the implementations are for unit of time required to put or take a
message? Are you looking for something like trying to put as many
messages as I can in 1 minute with each system and see which can push
the highest amount of messages through the system?

Since I will be doing some other runs tonight, feel free to make any
other suggestions, I don't have a ton of time to spend on this but if
there is anything easy enough to throw in I can try to add it to the
test.

[danm...@gmail.com]

Alright here are soome runs with 10,000 and 100,000 messages between
two EC2 instances...

updated post:
http://devver.net/blog/2008/06/ruby-messaging-shootout/

Quick summary of the most interesting parts below

10,000 messages, Remote Queue servers on EC2 (client and tests running
on a separate EC2 instance)

Queue type user system total real

MemoryQueue: 0.040000 0.000000 0.040000 ( 0.127432)
BeanstalkClientRemote: 0.390000 0.090000 0.480000 ( 7.646054)
RemoteStarlingQueue: 0.070000 0.020000 0.090000 ( 10.685410)

100,000 messages, Remote Queue servers on EC2 (client and tests
running on a separate EC2 instance)

Queue type user system total real

MemoryQueue: 0.260000 0.040000 0.300000 ( 0.677368)
BeanstalkClientRemote: 3.200000 0.940000 4.140000 ( 76.989950)
RemoteStarlingQueue: 0.820000 0.240000 1.060000 (110.507879)

BeanstalkClientRemote:::::
mean time for put : 0.000274
std dev for put: 0.002125
mean time for take: 0.000531
std dev for take: 0.003302
put mean is 14.6932435272862 slower than MemoryQueue
take mean is 30.0050748059956 slower than MemoryQueue

RemoteStarlingQueue:::::
mean time for put : 0.000592
std dev for put: 0.006346
mean time for take: 0.000577
std dev for take: 0.004349
put mean is 31.7037894886494 slower than MemoryQueue
take mean is 32.5781596463523 slower than MemoryQueue

[Dustin]

That looks pretty good. starling is keeping up on gets at least.

Any chance you could try it with my ``performance'' branch?

http://github.com/dustin/beanstalkd/commits/performance

[Blaine Cook]

Out of curiosity, are you running the gem version of Starling (0.9.3)?
There are two things in that code that make it significantly slower
than more recent versions on github ( http://github.com/starling/starling/tree/master
); first, an fsync in the persistence code, and second the newer
versions switched from a threading to evented model. In any case, the
newest version should be 2-4 times faster (especially for put) than
0.9.3.

Also, it would be interesting to see benchmarks that factor in
concurrency, but it's great to see these comparisons.

cheers,

b.

[danm...@gmail.com]

I am pretty busy with trying to move development forward for Devver,
but I will see if I get some time and can run on the performance
branch sometime... If so I might have to compare it with the newer gem
version of Starling as well since it seems to be a lot faster.

Blaine,
That is very interesting I wasn't aware of the faster version that
hadn't made it to gems. I was running on the current Gem version
(0.9.3). Some concurrency tests would be good, as our production
system runs with up to 20 EC2 instances currently, and it would be
good to know if the constant hammering from multiple machines effects
the performance of these systems at all. I will keep that in mind for
the future as well.

I can report that beanstalk runs like a champ when we are on 20 EC2
instances, and our messaging overhead for overall workload has already
been reduced so much that we don't really worry about it anymore.

Dan