Adding timeouts to Riddle?

[Jonathan del Strother]

Heya,

I ran into some problems yesterday where searchd froze up, leading to all my rails instances getting permanently stuck waiting on a response from Thinking Sphinx.

It would be nice if it were possible to specify a sphinx timeout for this sort of problem.  I had a quick hack around with the riddle source, adding timeout blocks around the TCPSocket initialization, and using recv_nonblock instead of recv, which seemed to work ok & suggested it might actually be possible.  However, I'd be the first to admit I probably don't know enough about sphinx, riddle, or even socket programming to say for sure that it's the right approach.  Has this sort of thing already been discussed?  I could probably neaten things up and publish a branch if there's any interest in it.

[Pat Allan]

Hi Jonathan

While I know plenty about Riddle, and a decent amount about Sphinx, working with sockets is not something I know a notable amount about, so your guess is as good as mine when it comes to that. It'd be great to see your patch if you have some time - and bonus points if it has tests :)

Cheers

--
Pat

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/9Ho8lZU7AjYJ.
> To post to this group, send email to thinkin...@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.

[Sam Goldstein]

Jonathan,

I believe riddle and ts have timeout functionality already that would solve this problem.

https://github.com/freelancing-god/thinking-sphinx/blob/master/lib/thinking_sphinx/configuration.rb#L253
https://github.com/freelancing-god/thinking-sphinx/blob/master/spec/thinking_sphinx/configuration_spec.rb#L269
https://github.com/freelancing-god/riddle/blob/master/lib/riddle/client.rb#L495
https://github.com/freelancing-god/riddle/blob/master/spec/unit/client_spec.rb#L206

I think you can put a `timeout: <seconds>` in your sphinx.yml.

~sam

--

[Jonathan del Strother]

On Sunday, 11 December 2011 00:22:10 UTC, Sam Goldstein wrote:

Jonathan,

I believe riddle and ts have timeout functionality already that would solve this problem.

...

I think you can put a `timeout: <seconds>` in your sphinx.yml.

Huh.  I looked at that and discounted it because I was under the impression that ruby's Timeout wouldn't actually raise a timeout while opening/receiving from that socket.  I even tried it out in irb while I was debugging the server problem and came to the same conclusion.  And yet... it seems to work totally fine now.

I think it could be argued that we ought to use recv_nonblock & timeout when making requests in https://github.com/freelancing-god/riddle/blob/master/lib/riddle/client.rb#L637, just in case the connection opens fine but searchd goes away before a request is made, but the existing timeout looks like it will cover 99% of the problem.

I'll try adding a timeout to our sphinx.yml and see what happens over the next few days.

-Jonathan