Ruby on Rails - Setting up a Solr server with Sunspot for production

[Joe]

I've recently purchased a linux server VPS and have managed to get my
rails 3.1 application online and running using apache and passenger.
It runs itself whenever the apache server starts and although I'm not
sure totally why or how this works - it works. My rails application
uses Sunspot for searching through it's contents - however this has
caused an issue, as it isn't as simple as running "rake
sunspot:solr:start RAILS_ENV=production" when in production.

It seems to be pretty poorly documented how to setup a Solr server and
get it working with a rails application - however I've had a go. I
followed the instructions here to setup a Tomcat/Solr server:
https://github.com/outoftime/sunspot/wiki/Configure-Solr-on-Ubuntu,-the-quickest-way
However I'm not quite sure what I'm supposed to do with it.. going to :
8080/solr and :8983:/solr give me weird admin panels which I don't
seem to be able to do anything interesting with, but I'm sure that
these URLs are important.

When running my rails app - I get the following error on pages with
the search integrated:

Connection refused - connect(2)

Which I think indicates that it can't access the server properly or
something. I've been googling and trying things for hours - but to no
avail. Could anyone help me get this working?

Please Help,

Thanks In Advance,

Joe

[Hassan Schroeder]

On Mon, Sep 5, 2011 at 8:16 AM, Joe <you...@dev-hq.co.uk> wrote:

> It seems to be pretty poorly documented how to setup a Solr server and
> get it working with a rails application - however I've had a go.

> When running my rails app - I get the following error on pages with

> the search integrated:
>
> Connection refused - connect(2)
>
> Which I think indicates that it can't access the server properly or
> something.

No kidding :-)

What does your config/sunspot.yml look like? Assuming solr/tomcat
is really running and you're accessing those URLs in a browser, it
sounds like your app is trying to connect to the wrong port.

--
Hassan Schroeder ------------------------ hassan.s...@gmail.com
http://about.me/hassanschroeder
twitter: @hassan

[Joe]

config/sunspot.yml:
production:
solr:
hostname: localhost
port: 8983
path: '/solr/'
log_level: WARNING

development:
solr:
hostname: localhost
port: 8982
log_level: INFO

test:
solr:
hostname: localhost
port: 8981
log_level: WARNING


On Sep 5, 4:37 pm, Hassan Schroeder <hassan.schroe...@gmail.com>
wrote:

> On Mon, Sep 5, 2011 at 8:16 AM, Joe <yout...@dev-hq.co.uk> wrote:
> > It seems to be pretty poorly documented how to setup a Solr server and
> > get it working with a rails application - however I've had a go.
> > When running my rails app - I get the following error on pages with
> > the search integrated:
>
> > Connection refused - connect(2)
>
> > Which I think indicates that it can't access the server properly or
> > something.
>
> No kidding  :-)
>
> What does your config/sunspot.yml look like? Assuming solr/tomcat
> is really running and you're accessing those URLs in a browser, it
> sounds like your app is trying to connect to the wrong port.
>
> --

> Hassan Schroeder ------------------------ hassan.schroe...@gmail.comhttp://about.me/hassanschroeder
> twitter: @hassan

[Hassan Schroeder]

On Mon, Sep 5, 2011 at 8:45 AM, Joe <you...@dev-hq.co.uk> wrote:
> config/sunspot.yml:
> production:
>  solr:
>    hostname: localhost
>    port: 8983
>    path: '/solr/'

OK, can you connect manually from a shell on the machine? To make
sure there's no firewall/iptables whatever issues.

[Joe]

Connect manually to what?
When running "curl localhost:8983/solr/" it returns the page with the
"Solr Admin" link on if that's what you mean.

On Sep 5, 5:05 pm, Hassan Schroeder <hassan.schroe...@gmail.com>
wrote:

> On Mon, Sep 5, 2011 at 8:45 AM, Joe <yout...@dev-hq.co.uk> wrote:
> > config/sunspot.yml:
> > production:
> >  solr:
> >    hostname: localhost
> >    port: 8983
> >    path: '/solr/'
>
> OK, can you connect manually from a shell on the machine? To make
> sure there's no firewall/iptables whatever issues.
>
> --

> Hassan Schroeder ------------------------ hassan.schroe...@gmail.comhttp://about.me/hassanschroeder
> twitter: @hassan

[Matt Jones]

On Sep 5, 11:45 am, Joe <yout...@dev-hq.co.uk> wrote:
> config/sunspot.yml:
> production:
>   solr:
>     hostname: localhost
>     port: 8983
>     path: '/solr/'

I think (based on some quick testing against Sunspot's default Solr
instance) that the trailing / here is blowing things up. The
underlying code takes the given path and adds things like '/select' to
it, which creates double slashes in the URL. For instance, this
request succeeds (in development mode, so port 8982):

GET http://localhost:8982/solr/select?q=*:*

but this returns a 404:

GET http://localhost:8982/solr//select?q=*:*

Not sure how this is turning into a connect failure, but it's worth
checking out...

--Matt Jones

[Joe]

Removing the trailing slash on '/solr/' and restarting the apache2
server unfortunately still results in the connection issue.
Going to http://localhost:8983/solr/select?q=*:* results in an XML
file as shown:

<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">164</int>
<lst name="params">
<str name="q">*:*</str>
</lst>
</lst>
<result name="response" numFound="0" start="0"/>
</response>

And when trying to run the reindex command: rake sunspot:solr:reindex
RAILS_ENV=production
I get the following error from the terminal:

rake aborted!
Solr Response: Internal Server Error


Thanks for all of your fast replies - hopefully I get can this issue
sorted sometime soon!
Any further help would be much appreciated!

On Sep 6, 5:50 pm, Matt Jones <al2o...@gmail.com> wrote:
> On Sep 5, 11:45 am, Joe <yout...@dev-hq.co.uk> wrote:
>
> > config/sunspot.yml:
> > production:
> >   solr:
> >     hostname: localhost
> >     port: 8983
> >     path: '/solr/'
>
> I think (based on some quick testing against Sunspot's default Solr
> instance) that the trailing / here is blowing things up. The
> underlying code takes the given path and adds things like '/select' to
> it, which creates double slashes in the URL. For instance, this
> request succeeds (in development mode, so port 8982):
>

> GEThttp://localhost:8982/solr/select?q=*:*

>
> but this returns a 404:
>

> GEThttp://localhost:8982/solr//select?q=*:*

[Joe]

After talking more with Hassan and long sessions of Troubleshooting -
I managed to fix the issue.
I firstly deleted the sunspot.yml file in the config folder of my
rails application and replaced it with an initializer (sunspot.rb) in
my config/initalizers directory which specified the sunspot path:

Sunspot.config.solr.url = 'http://localhost:8983/solr'

After doing this I stopped getting the errors while in development,
although a reindex was required. When trying to reindex I then
encountered the error: "Solr Response: Internal Server Error"
At this stage I tried a variety of things to get it working - some/all
of these may or may not have had a bearing on the getting it to work.
I changed the file permissions (chmod, chown) of the files in my rails
application to make sure they were all fully accessible and writable
to by apache and solr, changed Java versions from OpenJDK to Oracle/
Sun JDK (sudo update-alternatives --config java) - I think this was
the main fix to this problem, closed down another server that was
running on the same port (plus closed down tomcat since it turns out
it wasn't being used).
After this - everything reindexed fine and it worked a treat!
Closing the other server that was running on the same port also fixed
an issue in which I got the "Connection refused - connect(2) " when
running is production mode despite it working in development.

Hope this helps other people that may be experiencing the same or
similar problems!