indexing not working (rotating successful)

[internetchris]

Hi Group,

I'm not sure why this isn't working. I have setup TS in the past without issue. I'm re-indexins my Rails app using a cronjob...

The cronjob log makes it looks like it's successful, if I run the script manually I get the following output.

Generating Configuration to /home/public_html/myapp.com/releases/20121015032144/config/production.sphinx.conf

Sphinx 2.0.5-release (r3308)

Copyright (c) 2001-2012, Andrew Aksyonoff

Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/home/public_html/myapp.com/releases/20121015032144/config/production.sphinx.conf'...

indexing index 'user_core'...

WARNING: collect_hits: mem_limit=0 kb too low, increasing to 13568 kb

collected 74 docs, 0.0 MB

sorted 0.0 Mhits, 100.0% done

total 74 docs, 2438 bytes

total 0.028 sec, 86026 bytes/sec, 2611.15 docs/sec

skipping non-plain index 'user'...

total 3 reads, 0.000 sec, 1.5 kb/call avg, 0.0 msec/call avg

total 9 writes, 0.000 sec, 1.1 kb/call avg, 0.0 msec/call avg

rotating indices: succesfully sent SIGHUP to searchd (pid=713).

The biggest thing I see here is that it's "rotating indices" and I don't remember seeing any of my previous apps doing this. In addition, I'm using a rackspace cloud mysql db, but that should make a difference if it's finding my data. I can finally get it to work after several attempts using "rake ts:rebuild", but that's not ideal. 

Here's my cronjob:

# m h  dom mon dow   command

*/1 * * * * cd /home/public_html/myapp.com/current && source /home/internetchris/.rvm/gems/rub-1.9.3-p194@global/bin/rake  && /usr/bin/rake RAILS_ENV=production ts:index

Here's my syslog output for the cronjob:

Oct 15 16:03:01 web01 CRON[16079]: (internetchris) CMD (cd /home/public_html/myapp.com/current && source /home/internetchris/.rvm/gems/rub-1.9.3-p194@global/bin/rake  && /usr/bin/rake RAILS_ENV=production ts:index)

Any ideas?

[Pat Allan]

Hi Chris

I'm not spotting anything immediately obvious that would cause this problem... but I guess the core issue is that Thinking Sphinx, within the context of cron, doesn't think that Sphinx is running. The fact that it's not consistent with this is definitely confusing.

Can you run us through the contents of your sphinx.yml file, if you have one?

--
Pat

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/p_xtiN0qcH4J.
> To post to this group, send email to thinkin...@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.

[internetchris]

Hi Pat,

I'm finally getting back to trying to figure this out. Currently, to get it to work I have to kill the process and re-index manually. I added a sphinx.yml file and put the following in the file..

production:

  mem_limit: 128M

  bin_path: '/usr/local/bin'

  enable_star: 1

  min_infix_len: 3

  indexed_models:

     - User

     - Gram

     - Photo

When running the task manually "outside of cron" it doesn't work either, but my output looks good.

using config file '/home/public_html/mysite.com/releases/20121103203038/config/production.sphinx.conf'...

indexing index 'gram_core'...

collected 259 docs, 0.1 MB

sorted 0.3 Mhits, 100.0% done

total 259 docs, 141938 bytes

total 0.213 sec, 663348 bytes/sec, 1210.43 docs/sec

skipping non-plain index 'gram'...

indexing index 'photo_core'...

collected 970 docs, 0.2 MB

sorted 0.3 Mhits, 100.0% done

total 970 docs, 155598 bytes

total 0.186 sec, 835084 bytes/sec, 5205.92 docs/sec

skipping non-plain index 'photo'...

indexing index 'user_core'...

collected 81 docs, 0.0 MB

sorted 0.0 Mhits, 100.0% done

total 81 docs, 2674 bytes

total 0.027 sec, 97966 bytes/sec, 2967.57 docs/sec

skipping non-plain index 'user'...

total 9 reads, 0.004 sec, 303.7 kb/call avg, 0.4 msec/call avg

total 30 writes, 0.014 sec, 198.1 kb/call avg, 0.4 msec/call avg

rotating indices: succesfully sent SIGHUP to searchd (pid=11651).

When I type "which searchd" I get this...

/usr/local/bin/searchd

ls -la on searchd looks like this..

-rwxr-xr-x  1 root root 12727895 Aug 23 02:27 searchd

When I run "search" this is my Sphinx Version..

Sphinx 2.0.5-release (r3308)

Copyright (c) 2001-2012, Andrew Aksyonoff

Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)

I hope this helps, I'm trying to figure it out. I'm now looking at my sphinx install as the culprit (possible permission issue?).

[internetchris]

I'm hesitant to say I have it working, but it seems to be. The only thing I noticed is that the owner of my script file (.sh)  was "root". I chown'ed the file and, stopped the service and then restarted everything. It seems to be working. I don't really know why it wouldn't have given me an error prior, but I won't argue with the fact it's working. I will monitor it closely over the next week.

[Pat Allan]

Hi Chris

Permissions could have certainly played a part in why things wouldn't work... but still, if you hit any further problems, let us know.

Cheers

--
Pat

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/wuO79te3_NEJ.