Ubuntu production issues starting thinking sphinx, TS3, Sphinx 2.0.5, Rails 3.2.xx

JS

unread,

Nov 1, 2013, 6:04:16 PM11/1/13

to thinkin...@googlegroups.com

Hello,

I'm trying to start deploy a rails 3.2.13 project to an ubuntu server and the issue below has me stumped:

When running:

RAILS_ENV=production rake ts:start

I am met with:

Failed to start searchd daemon. Check /rails root /log/production.searchd.log.

Failed to start searchd. Check the log files for more information.

Inside that files stated:

[Fri Nov 1 14:24:12.764 2013] [ 2018] listening on 127.0.0.1:9306

[Fri Nov 1 14:24:12.772 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:15.775 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:18.778 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:21.781 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:24.784 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:27.787 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:30.790 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:33.793 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:36.796 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:39.799 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:42.802 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:45.805 2013] [ 2018] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:24:48.809 2013] [ 2018] FATAL: bind() failed on 127.0.0.1: Address already in use

[Fri Nov 1 14:24:48.810 2013] [ 2017] Child process 2018 has been finished, exit code 1. Watchdog finishes also. Good bye!

At which point I created a config/thinking_sphinx.yml file and added the following, trying to specify a different port

production:

mysql41: 9312

When running this command again:

RAILS_ENV=production rake ts:start

I am again met with:

Failed to start searchd daemon. Check /rails root /log/production.searchd.log.

Failed to start searchd. Check the log files for more information.

The log this time stated: (please note that the port number has chaged)

[Fri Nov 1 14:32:58.118 2013] [ 2606] Child process 2607 has been forked

[Fri Nov 1 14:32:58.119 2013] [ 2607] listening on 127.0.0.1:9312

[Fri Nov 1 14:32:58.119 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:01.120 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:04.123 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:07.126 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:10.129 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:13.132 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:16.135 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:19.138 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:22.142 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:25.145 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:28.148 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:31.151 2013] [ 2607] bind() failed on 127.0.0.1, retrying...

[Fri Nov 1 14:33:34.154 2013] [ 2607] FATAL: bind() failed on 127.0.0.1: Address already in use

[Fri Nov 1 14:33:34.156 2013] [ 2606] Child process 2607 has been finished, exit code 1. Watchdog finishes also. Good bye!

I am using the following:

rails '3.2.13'

thinking-sphinx '3.0.5'

Postgres as my database (yes I also have the mysql2 gem installed)

Ubuntu 12.04.2 LTS (GNU/Linux 3.5.0-42-generic x86_64)

Sphinx 2.0.5-id64-release (r3308)

Apache2

phusion passenger

Please let me know if ive left out any relevant information or if there is some documentation you think I may have overlooked. Thanks in advance

Pat Allan

unread,

Nov 2, 2013, 1:27:28 AM11/2/13

to thinkin...@googlegroups.com

Hi JS

Looks like the rake task is running twice somehow. You’re not including the TS tasks in your Rakefile manually, are you?

Also, what happens if you run the rake task through bundle exec?

—

Pat

--
You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thinking-sphi...@googlegroups.com.
To post to this group, send email to thinkin...@googlegroups.com.
Visit this group at http://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/groups/opt_out.

JS

unread,

Dec 14, 2013, 12:01:45 AM12/14/13

to thinkin...@googlegroups.com

Pat,

What was stated above is no longer my exact issue however what I've currently running into may be related.

I deployed my code to a vps running ubuntu 12.04 via capistrano, requiring thinking_sphinx/capistrano. I received no errors when deploying, however thinking sphinx is not currently working in my production mode. Below I have the error that is return when trying to access a page that calls a thinking sphinx search. I have also included the results from several other commands that I think would be helpful in tracking down the issue. Could you please tell me if I'm doing something wrong, or if there is some documentation I have overlooked, thanks in advance.

Currently if I try to visit a page where thinking sphinx should return info I receive the following error:

@user_feed is a variable which is the returned value from a thinking sphinx search

ActionView::Template::Error (index user_core: syntax error, unexpected $end near ' *'):

75:

76: <br>

77:

78: <% if @user_feed.present? %>

79: <% @user_feed.each do |user| %>

80:

81: <div class="user_index_container" rel="<%= user.id %>">

When I run searchd I receive the following:

Sphinx 2.0.4-release (r3135)

FATAL: no readable config file (looked in /etc/sphinxsearch/sphinx.conf, ./sphinx.conf).

When I run which searchd:

/usr/bin/searchd

sudo service sphinxsearch start

returns:

Starting sphinxsearch:

Please create an /etc/sphinxsearch/sphinx.conf configuration file.

A template is provided as /etc/sphinxsearch/sphinx.conf.sample.

Pat Allan

unread,

Dec 16, 2013, 7:02:09 AM12/16/13

to thinkin...@googlegroups.com

Hi JS

It sounds like Sphinx is definitely running, though something’s slightly off with what’s being sent to Sphinx. Can you confirm which versions of Thinking Sphinx and the Riddle gem (if it’s not in Gemfile, it’ll be mentioned in Gemfile.lock) you’re using?

Also: I’d highly recommend upgrading Sphinx to a more recent version - 2.0.9 is the latest 2.0.x release. 2.1.x has changed a few things, it’s certainly useable, but you’ll need to use TS 3.0.6 and note the tweaks in the release notes:

https://github.com/pat/thinking-sphinx/releases/tag/v3.0.6

—

Pat

Joshua Shuster

unread,

Dec 16, 2013, 9:39:34 PM12/16/13

to thinkin...@googlegroups.com

Pat,

Versions as follows:

riddle (1.5.8)

thinking-sphinx (3.0.5)

Also I have upgraded to sphinx version:

Sphinx 2.0.9-id64-release (rel20-r4115) (which is the version I am running on my mac in development and it's working fine)

I am currently receiving these errors when running the following commands, which I believe could be helpful for zooming in on the problem. Also, you will notice that it's starting to look like the problem I originally posted, per your request, I tried to run commands with bundle exec and the results are included below. And to answer your previous question I am not including TS rake tasks in my rakefile. Please let me know if any of this helps or if you need additional information:

RAILS_ENV=production rake ts:index

runs fine

RAILS_ENV=production rake ts:start or RAILS_ENV=production bundle exec rake ts:start

Failed to start searchd. Check the log files for more information.

the production.searchd.log states the following errors:

[Tue Dec 17 02:12:38.502 2013] [19206] listening on 127.0.0.1:9306

[Tue Dec 17 02:12:38.502 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:12:41.505 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:12:44.508 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:12:47.511 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:12:50.515 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:12:53.518 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:12:56.521 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:12:59.524 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:13:02.527 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:13:05.531 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:13:08.534 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:13:11.537 2013] [19206] bind() failed on 127.0.0.1, retrying...

[Tue Dec 17 02:13:14.540 2013] [19206] FATAL: bind() failed on 127.0.0.1: Address already in use

[Tue Dec 17 02:13:14.542 2013] [19205] Child process 19206 has been finished, exit code 1. Watchdog finishes also. Good bye!

RAILS_ENV=production rake ts:stop

searchd is not currently running.

Stopped searchd daemon (pid: ).

searchd

Sphinx 2.0.9-id64-release (rel20-r4115)

using config file '/etc/sphinxsearch/sphinx.conf'...

WARNING: compat_sphinxql_magics=1 is deprecated; please update your application and config

FATAL: failed to lock pid file '/var/run/sphinxsearch/searchd.pid': Resource temporarily unavailable (searchd already running?)

'/var/run/sphinxsearch/searchd.pid' contains:

19011

Thank You,

Joshua Shuster
JoshuaGreg...@gmail.com

--
You received this message because you are subscribed to a topic in the Google Groups "Thinking Sphinx" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/thinking-sphinx/8d8tVO1h4xo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to thinking-sphi...@googlegroups.com.

Pat Allan

unread,

Dec 22, 2013, 2:11:18 AM12/22/13

to thinkin...@googlegroups.com

Sorry for the slow response Joshua.

It looks like the daemon’s already running - which is why the start is failing. Perhaps it’s worth running ts:stop and then ts:start again? Also, you can use `ps aux | grep searchd` to find the running processes and perhaps kill them manually if required.

Starting searchd with no arguments isn’t going to do much - TS generates configuration files into your app’s config directory, with naming like production.sphinx.conf (or whatever the appropriate environment is), and that’s not one of Sphinx’s default configuration locations. If you want to try start it manually, then `searchd —config config/production.sphinx.conf` should work - but that’s pretty much what the rake task runs.

But yeah: the log is saying that something’s already using port 9306, and it’ll almost certainly be an existing Sphinx daemon with the PID 19011.