MemCache can talk to the Starling server but Starling can't?

[Morgan Grubb]

I've got a problem where the Starling client either explodes or hangs.
With the latest versions of Starling/Fiveruns memcache client

Starting up a starling client server

morgan@blackpower:~/classifieds2$ sudo starling -v -d
Starting at 127.0.0.1:22122.
I, [2009-03-13T11:23:53.814998 #5675] INFO -- : Starling STARTUP on
127.0.0.1:22122

Using script/console in Rails 2.3.1

>> require 'starling'
=> ["Starling"]
>> s = Starling.new 'localhost:22122'
=> MemCache: 1 servers, ns: nil, ro: false
>> s.set 'foo', 'bar'
TypeError: can't dup NilClass
from /usr/lib/ruby/gems/1.8/gems/starling-starling-0.9.9/lib/
starling.rb:137:in `dup'
from /usr/lib/ruby/gems/1.8/gems/starling-starling-0.9.9/lib/
starling.rb:137:in `get_server_for_key'
from /usr/lib/ruby/gems/1.8/gems/activesupport-2.3.1/lib/
active_support/vendor/memcache-client-1.6.5/memcache.rb:669:in
`request_setup'
from /usr/lib/ruby/gems/1.8/gems/activesupport-2.3.1/lib/
active_support/vendor/memcache-client-1.6.5/memcache.rb:638:in
`with_server'
from /usr/lib/ruby/gems/1.8/gems/activesupport-2.3.1/lib/
active_support/vendor/memcache-client-1.6.5/memcache.rb:283:in `set'
from /usr/lib/ruby/gems/1.8/gems/starling-starling-0.9.9/lib/
starling.rb:76:in `set'
from (irb):4

>> m = MemCache.new 'localhost:22122'
=> MemCache: 1 servers, ns: nil, ro: false
>> m.set 'foo', 'bar'
=> "STORED\r\n"
>> m.get 'foo'
=> "bar"
>> m.get 'foo'
=> nil


Using irb

>> require 'starling'
=> true
>> s = Starling.new 'localhost:22122'
=> MemCache: 1 servers, 1 buckets, ns: nil, ro: false
>> s.set 'foo', 'bar'
=> "STORED\r\n"
>> s.get 'foo'
=> "bar"
>> s.get 'foo'
[hangs, never comes back]

>> m = MemCache.new 'localhost:22122'
=> MemCache: 1 servers, 1 buckets, ns: nil, ro: false
>> m.set 'foo', 'bar'
=> "STORED\r\n"
>> m.get 'foo'
=> "bar"
>> m.get 'foo'
=> nil

Also, -v doesn't seem to increase the Starling logging one iota. I was
hoping to be seeing SET 'foo' and GET 'foo' and whatnot but it
displays nothing.

Can anybody shed some light on what I'm doing wrong here?


Cheers,
Morgan.

[Harm]

I'm experiencing the same thing with Rails 2.3.2 and the Starling lib
0.9.9. Did you find any solution?

[Chris Nolan.ca]

No help on the main issue but just wanted to comment on:

>>  require 'starling'
=> true
>> s = Starling.new 'localhost:22122'
=> MemCache: 1 servers, 1 buckets, ns: nil, ro: false
>> s.set 'foo', 'bar'
=> "STORED\r\n"
>> s.get 'foo'
=> "bar"
>> s.get 'foo'
[hangs, never comes back]

This is what it's supposed to do -- the .get waits for an entry on the queue.  If you want it to behave like memcache, use .fetch instead.

Chris Nolan.ca

On Tue, Mar 17, 2009 at 08:58, Harm <harm...@gmail.com> wrote:

[Morgan Grubb]

Thanks for that. At least it works like it's supposed to in IRB. :)


Cheers.

> On Tue, Mar 17, 2009 at 08:58, Harm <harmaa...@gmail.com> wrote:

[Morgan Grubb]

Not yet, I'm afraid. Trying to find some more time to look into it.

Cheers.

[Dharmarth]

I am getting the same issue with starling 0.9.8 and memcache-client
1.6.5 (& 1.7.0 as well).

irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'starling'
[memcache-client] Could not load SystemTimer gem, falling back to
Ruby's slower/unsafe timeout library: no such file to load --
system_timer
=> true
irb(main):003:0> s = Starling.new 'localhost:22122'

=> <MemCache: 1 servers, ns: nil, ro: false>

irb(main):004:0> s.set('foo', 'bar')

TypeError: can't dup NilClass

from /var/lib/gems/1.8/gems/starling-0.9.8/lib/starling.rb:94:in `dup'
from /var/lib/gems/1.8/gems/starling-0.9.8/lib/starling.rb:94:in
`get_server_for_key'
from /var/lib/gems/1.8/gems/memcache-client-1.7.0/lib/memcache.rb:
878:in `request_setup'
from /var/lib/gems/1.8/gems/memcache-client-1.7.0/lib/memcache.rb:
843:in `with_server'
from /var/lib/gems/1.8/gems/memcache-client-1.7.0/lib/memcache.rb:
333:in `set'
from /var/lib/gems/1.8/gems/starling-0.9.8/lib/starling.rb:34:in `set'
from (irb):4

It looks to me, class Starling overrides “get_server_for_key” method
of class MemCache, and uses an instance variable named
"@buckets"(starling-0.9.8/lib/starling.rb:94). Unfortunately, this
particular instance variable is no longer available in newer versions
on memcache-client (1.6.2 onwards). Earlier "@buckets" was used to
pick a server to handle the request, now "@continuum"(instance of
class Continuum) is used to do the same.

Could someone please suggest how I can proceed to fix this? One option
can be to override “initialize” method as well in the Starling class,
and specifically fill the @buckets array after calling super. This
would ensure starling runs with both older and newer versions of
memcache-client.

Thanks,
Dharmarth
Impetus Technologies

[lampy]

I also had the same problem. It looks like someone has fixed this
with their starling fork:

http://github.com/bricooke/starling/commit/dd35b801a2cca0cc7a80ee6e45b65cbd96adb144

I haven't put it into production yet but it seems to do the trick.

[Miggles]

Out of curiosity, how did you install it? I can't even find it using gem:

morgan@blackpower:~$ sudo gem install bricooke-starling

ERROR:  could not find gem bricooke-starling locally or in a repository

morgan@blackpower:~$ sudo gem search starling -r

*** REMOTE GEMS ***

advany-starling (0.9.8)

colinsurprenant-starling (0.9.8)

ctcherry-starling (0.9.10)

fiveruns-starling (0.9.7.6)

mmmurf-starling (0.9.8)

ryana-starling (0.9.8)

starling (0.9.8)

starling-starling (0.9.9)

timshadel-starling (0.9.8.200810061510)

Am I missing a trick here?

Cheers.

[Chris Cherry]

Add the github gem source first.

gem sources -a http://gems.github.com

---

From: Miggles
Date: Sun, 22 Mar 2009 15:14:52 +0100
To: <starl...@googlegroups.com>
Subject: Re: MemCache can talk to the Starling server but Starling can't?

[Miggles]

I've already done that bit. You might notice all the other github gems in the search results for starling. The one missing is bricooke-starling.

Cheers.

[Neha Mittal]

I followed the commands from this link
http://playtype.net/past/2008/10/2/workling_version_03_released/
and it worked.
Maybe u can also give it a try.

--
Regards
Neha
+1-201-850-8583

[lampy]

> Out of curiosity, how did you install it? I can't even find it using gem:

I actually modified the starling.rb file in my gems directory directly
as a quick fix.

His fork doesn't show up as a gem for me either. It must be a github
choice he didn't set. You could probably fork from him and check off
that option?

> morgan@blackpower:~$ sudo gem install bricooke-starling
> ERROR:  could not find gem bricooke-starling locally or in a repository
>
> morgan@blackpower:~$ sudo gem search starling -r
>
> *** REMOTE GEMS ***
>
> advany-starling (0.9.8)
> colinsurprenant-starling (0.9.8)
> ctcherry-starling (0.9.10)
> fiveruns-starling (0.9.7.6)
> mmmurf-starling (0.9.8)
> ryana-starling (0.9.8)
> starling (0.9.8)
> starling-starling (0.9.9)
> timshadel-starling (0.9.8.200810061510)
>
> Am I missing a trick here?
>
> Cheers.
>

> On Fri, Mar 20, 2009 at 6:32 PM, lampy <lampligh...@gmail.com> wrote:
>
> > I also had the same problem.  It looks like someone has fixed this
> > with their starling fork:
>

> >http://github.com/bricooke/starling/commit/dd35b801a2cca0cc7a80ee6e45...

[Miggles]

I did consider that but had already done exactly what you did: modify starling-starling as a quick fix. :)

[Dharmarth]

Seth,

For getting starling to work with newer versions of memcache-client
(1.6.2 onwards), the fix in bricooke’s fork ignores “@buckets” array.
I don’t have a great understanding of starling internals at this
point, but it seems to me it might not be optimal and could impact
weight based load balancing logic.

In my fork, I have tried to fix the issue by overriding ‘initialize’
method of class MemCache to explicitly fill the “@buckets” array. This
fix ensures starling works with all (including older) versions of
memcache-client.

The changes can be found here:
http://github.com/dharmarth/starling/commit/98f1e5243cb01c3bfeea3dc6938bc7e2a5bf9e0b

I have also sent a pull request to you for these.

Dharmarth,
Impetus Technologies

On Mar 20, 10:32 pm, lampy <lampligh...@gmail.com> wrote:
> I also had the same problem.  It looks like someone has fixed this
> with their starling fork:
>

> http://github.com/bricooke/starling/commit/dd35b801a2cca0cc7a80ee6e45...

[Harm]

I think we can safely say the starling-starling gem on Github is dead
(ish). None of the forks on Github seem to work with the new Memcache
client v 1.7.1.

On Mar 24, 12:25 pm, Dharmarth <dharmarth.s...@impetus.co.in> wrote:
> Seth,
>
> For getting starling to work with newer versions of memcache-client
> (1.6.2 onwards), the fix in bricooke’s fork ignores “@buckets” array.
> I don’t have a great understanding of starling internals at this
> point, but it seems to me it might not be optimal and could impact
> weight based load balancing logic.
>
> In my fork, I have tried to fix the issue by overriding ‘initialize’
> method of class MemCache to explicitly fill the “@buckets” array. This
> fix ensures starling works with all (including older) versions of
> memcache-client.
>

> The changes can be found here:http://github.com/dharmarth/starling/commit/98f1e5243cb01c3bfeea3dc69...

[Dharmarth]

Harm,

Could you please tell specific issue you are facing with this fork
(http://github.com/dharmarth/starling/tree/master)?

Dharmarth,
Impetus Technologies

[Dharmarth]

Please find the working link as http://github.com/dharmarth/starling/tree/master

[Harm]

I'm using a slightly hacked version(see this gist: http://gist.github.com/88687)
of the original starling but it amounts to the same:

MemCache::MemCacheError (No servers available (all dead)):
/Library/Ruby/Gems/1.8/gems/starling-starling-0.9.9/lib/starling.rb:
145:in `get_server_for_key'
/Library/Ruby/Gems/1.8/gems/memcache-client-1.7.1/lib/memcache.rb:
876:in `request_setup'
/Library/Ruby/Gems/1.8/gems/memcache-client-1.7.1/lib/memcache.rb:
841:in `with_server'
/Library/Ruby/Gems/1.8/gems/memcache-client-1.7.1/lib/memcache.rb:
333:in `set'
/Library/Ruby/Gems/1.8/gems/starling-starling-0.9.9/lib/starling.rb:
76:in `set'
lib/starling_managers/queue_manager.rb:29:in `enqueue_analyses'
app/controllers/abstract_suggestions_controller.rb:21:in `create'

Note that it works with the memcache-client lib 1.7.0 but not with the
1.7.1.

I am not that keen on the direct access on the instance variables and
the faking of the @buckets var. I just spoke with the dev of memcach-
client and he tells me the whole buckets thing is now redundant.

<10 minutes later>

Dharmarth I downloaded your fork and that experiences the same
problems. And I am wondering whether I do something wrong as nearly
all (that is 5 out of 11) specs fail.

best regards,
Harm


On Apr 1, 2:15 pm, Dharmarth <dharmarth.s...@impetus.co.in> wrote:
> Please find the working link ashttp://github.com/dharmarth/starling/tree/master

[Seth Fitzsimmons]

That's fair.

I haven't been in a position to apply any patches (sorry, particularly
for not owning up to this sooner). I'm happy to hand over the
credentials for the starling account (as well as the rubyforge account
for official releases) to whomever is willing to commit to becoming a
more official maintainer.

seth

[Andrei Maxim]

Hi,

I've had the very same problem that you guys were experiencing. I've
forked bricooke's version so I'd be able to set the RubyGem option, I
updated the README a bit so newcomers will not fall into the get/fetch
trap and changed the memcache-client dependency so it will use 1.7.0
or later. So all that's standing between you and a working Starling
server is:

sudo gem install xhr-starling

I know people aren't too fond of forking or branching, but I needed a
working Starling gem for my contract work and that was the fastest
way. I've sent a pull request to the main starling repository,
hopefully they will find it worthy.

I plan to take a closer look at the whole memcache-client/starling
issue and see if there's a better way to do things. Is there a more
detailed TODO list available so maybe I'll take a stab at implementing
those tasks?

Andrei

[Harm]

I took the liberty of properly fixing the specs and the lib. My fork
now works with Memcache-client 1.7.1. The latest, the greatest.

What I've done:
- removed any calls involving direct access to instance variables
- let the memcache client lib handle the locking and unlocking of
threads.
- updated the README to reflect the current state of affairs
- removed the now silly dependency on the fiveruns memcache client.
- removed any reference to the now deprecated @buckets.

You can find it here: http://github.com/haarts/starling/tree/master/

@Andrei Does your fork work with 1.7.1? The @server/@bucket hack
stopped working in 1.7.1.

Harm

> ...
>
> read more »

[Harm]

What is missing BTW is 1.9 compatibility. I haven't tested that, it
might not even be a problem.

> ...
>
> read more »

[Andrei Maxim]

@Harm, I haven't tested with 1.7.1, but I'll add it for my TODO list.
Thanks for telling me about the changes to the memcache-client
library. I think I'll need to start following their development from
now on.

> ...
>
> read more »

[Dharmarth]

Harm,
I tested out your fork on Ruby 1.8.6 (and 1.8.7), with the old
memcache-client 1.5.0 and the latest one 1.7.1. It's working perfect.
Nice work.
Also, i tested it out on Ruby 1.9.1, and it works fine on that as
well.

Dharmarth
Impetus Technologies

> ...
>
> read more »

[johan pretorius]

I'm having the exact same problem.

The funny thing is, it used to work ( .fetch no longer worked in a
specific script). I also have to use "require 'rubygems' " before
starling will load.

Tried all the forks listed below, with no success.

Any help or pointers would be appreciated...

some versions (on OpenSUSE 11.0, btw):

ruby 1.8.6 (yes, I know ...)
Rails 2.2.2

starling (0.9.8)
starling-starling (0.10.0)
memcache-client (1.7.2)
memcached (0.14, 0.13) (used 0.13, installed later version)

and the irb session:

irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'starling'

=> true
irb(main):003:0>

irb(main):004:0*
irb(main):005:0* starling = Starling.new('127.0.0.1:22122')

=> <MemCache: 1 servers, ns: nil, ro: false>

irb(main):006:0> starling.set('my_queue', 12345)

TypeError: can't dup NilClass

from /home/johan/.gem/ruby/1.8/gems/starling-0.9.8/lib/
starling.rb:94:in `dup'
from /home/johan/.gem/ruby/1.8/gems/starling-0.9.8/lib/
starling.rb:94:in `get_server_for_key'
from /usr/local/lib/ruby/gems/1.8/gems/memcache-client-1.7.2/
lib/memcache.rb:876:in `request_setup'
from /usr/local/lib/ruby/gems/1.8/gems/memcache-client-1.7.2/
lib/memcache.rb:841:in `with_server'
from /usr/local/lib/ruby/gems/1.8/gems/memcache-client-1.7.2/
lib/memcache.rb:333:in `set'
from /home/johan/.gem/ruby/1.8/gems/starling-0.9.8/lib/
starling.rb:34:in `set'
from (irb):6

[Andrei Maxim]

Hi Johan,

The problem you're having is because you're working with starling 0.9.8 and memcache-client 1.7. You should use the Github starling gem that's at version 0.10.0. The simplest way would be to uninstall the starling 0.9.8 gem from your system or you could use an explicit require:

require "rubygems"

gem "starling-starling"

require "starling"

starling = Starling.new('127.0.0.1:22122')

starling.set('my_queue', 12345) #=> "STORED\r\n"

Andrei Maxim
http://artfulco.de

[johan pretorius]

Andrei,

> The problem you're having is because you're working with starling 0.9.8 and
> memcache-client 1.7. You should use the Github starling gem that's at
> version 0.10.0. The simplest way would be to uninstall the starling 0.9.8
> gem from your system

That is the weird thing - I'm not installing starling 0.9.8 (almost
seems as if installing starling-starling did that?)

And even if it shows up in the gem list, I cannot uninstall it. I'm
going to do some gem cleanup now to see whether that helps.

Should I also uninstall memcache-client, surely that is required?

> or you could use an explicit require:
> require "rubygems"
> gem "starling-starling"
> require "starling"

I will try that, thanks.

--
rgrds,

Johan

[Andrei Maxim]

On Wed, Apr 29, 2009 at 10:37 AM, johan pretorius <johan.p...@gmail.com> wrote:

Andrei,

> The problem you're having is because you're working with starling 0.9.8 and
> memcache-client 1.7. You should use the Github starling gem that's at
> version 0.10.0. The simplest way would be to uninstall the starling 0.9.8
> gem from your system

That is the weird thing - I'm not installing starling 0.9.8 (almost
seems as if installing starling-starling did that?)

You did install it or some gem that lists starling as a dependency at some point in time, that's why it's showing up in the gems list.

 

Should I also uninstall memcache-client, surely that is required?

You don't need to do that. 

 

Andrei

[johan pretorius]

Andrei,

Thanks for the response.

> You did install it or some gem that lists starling as a dependency at some
> point in time, that's why it's showing up in the gems list.

It must be the latter, will try your solution as well. Will post the
results once done.

--
rgrds,

Johan

[johan pretorius]

Andrei,

Thanks for the suggestions and help, in the end it was caused by some
gems floating around in my home folder.

It's sorted now, I'm having some fun with starling-starling at last :-)

--
rgrds,

Johan