"No response from searchd" random error

[mhodgson]

Hello all,

I'm being plagued with a very strange problem. I've had Thinking
Sphinx setup and running for a long time, and have always had this
problem. Randomly, every few searches, TS throws a "No response from
searchd" error. When attempting the exact same search even a split
second later, the search comes back fine. This happens all the time
and is very frustrating. Has anyone ever had anything like this happen
to them? Like I said, it has nothing to do with the actual query
strings or parameters, since you can try it again and it works right
away.

Thanks for the help,
Matt

[Pat Allan]

Hi Matt

This isn't the first I've heard of this, it's just not been a priority
so far. I've had someone submit a patch for Riddle which may handle
the error, make a few attempts to connect, but haven't had time to
merge it in yet.

So: you're not alone, and hopefully it can be fixed soon.

--
Pat

[mhodgson]

Thanks Pat, I appreciate it.

Again, thanks for all your hard work.

-Matt

[Pat Allan]

Hi Matt

The latest in GitHub includes the patch to Riddle (it retries failed
connections five times) - v1.2.12. Can you give it a shot, see if it
fixes the problem for you?

Cheers

--
Pat

[Aaron Gibralter]

I noticed that I started getting the "No response from searchd" error when I have the following setup:

* development environment

* Rails 2.3.10

* Thin

And I change the following in config/environments/development.rb: `config.cache_classes = true`

Any thoughts?

[Pat Allan]

Hi Aaron

I've never seen this in production or development (but going by emails here, plenty of others have). How often is it happening for you? Is it easy to reproduce?

I'm wondering if it's related to the cache_class setting, and maybe that leads to an invalid query being sent to Sphinx. If you can make it happen regularly, perhaps it's worth adding some debug output to TS to find out what query is being generated.

Cheers

--
Pat

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> To post to this group, send email to thinkin...@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.

[Aaron Gibralter]

Hi Pat,

Thanks for the quick reply!

I can get the exception on every request. These are the ingredients of the problem:

* running thin in development mode: `RAILS_ENV=development bundle exec thin start` (perhaps Thin's evented server model causes the connection to searchd to go dead?)

* having a index defined on my user model

* using authlogic... when I'm logged in I guess authlogic always updates the user row on every request with the timestamp of the last request... or something like that.

* having `config.cache_classes = true` in my environment...

Changing any of the last 3 things causes the problem to go away; i.e. if I comment out the index on my user model, or if I am not logged in, or if I turn off cache_classes everything works ok: I can navigate to pages that show search results. It seems like the problem has to do with either an update_attribute or a delta index...

[Pat Allan]

Huh... so it's on the update, it seems. Can you provide the stack trace? And your define_index block?

--
Pat

[Aaron Gibralter]

Hmm weird -- I can't seem to recreate it on my mac book pro... I will try again when I get to my imac at work tomorrow.

[Aaron Gibralter]

Alright here is the stack trace: https://gist.github.com/e53720939e66367a23e1

[Pat Allan]

Hi Aaron

Sorry for the delay. Here's what I can tell from that stack trace:
* the record you're using has delta set to true.
* Thinking Sphinx is searching for that record in the delta index...

However, delta indexing hasn't happened at that point (if this is the first edit), so the record doesn't exist in the delta index. So it *should* just return an empty result set, but it's failing hard.

Can you try something for me? After running a full index (so no delta flags are set), grab a user instance, and run the following:
User.search_for_id(user.sphinx_document_id, 'user_delta')

Does it fail? Or return false?

--
Pat

[Aaron Gibralter]

That returns false for me.

[Pat Allan]

Hi Aaron

I've just made a change to TS which should hopefully work around this. It'll be a bit fiddly, but can you grab my latest source, and run the following commands:

bundle
rake build
gem install pkg/thinking-sphinx-1.4.3.gem --local

I just want to confirm it fixes this problem before pushing out a new gem release. Thanks!

--
Pat

[Aaron Gibralter]

Alright so I installed from the source and

User.search_for_id(user.sphinx_document_id, 'user_delta')

is still false.

My app works now with cache_classes = true. But... I decided to double check an reinstall the rubygems 1.4.3 version of TS and it still worked... so it the problem must have had to do with the state of the indexes... I'm not sure how to recreate the original problem.

What did you change in the code?

[Pat Allan]

I didn't do much - just the one commit since 1.4.3:
https://github.com/freelancing-god/thinking-sphinx/commit/3cc67600da484481ac304237b41d287d9e19a0b3

Stripped out some unused methods (and their specs), reworked one to do what we need, with a rescue for the error you're getting.

False is what I'd expect from that code, so that's fine... am defaulting to true, though - as there's other rescue blocks that will handle these situations.

--
Pat

[Aaron Gibralter]

Cool, well looks good to me. Thank you again for the help!

[Pat Allan]

No worries :)

--
Pat