Re: [ts] Thinking Sphinx skipping models when creating development.sphinx.conf

[Pat Allan]

Hi Aaron

Can you share the contents of your sphinx.yml file? Sounds like it's not quite formatted correctly...

That said, all models should be there - this is certainly an odd problem. Wonder if it's related to the project rather than Lion...

--
Pat

On 12/08/2011, at 5:26 AM, Aaron B. Russell wrote:

> I recently wiped my laptop to install Mac OS X Lion, and am having some trouble setting up Thinking Sphinx in my development environment again. This situation has been replicated on another developer's laptop while he was trying to set up the same project.
>
> When trying to run "rake thinking_sphinx:index" for the first time and letting it autocreate my development.sphinx.conf, I'm seeing certain models being completely skipped. Not mentioned in the output of "rake thinking_sphinx:index", and no mention of them in the resulting development.sphinx.conf ... it's as if I've failed to mention that those models should be indexed. These models were previously being indexed... infact they still are in our production environment, but of course we don't want to remove the production.sphinx.conf there and try rebuilding because it seems quite likely that our production environment will just break the same way it has on our development machines.
>
> Searching for people having similar problems (both here in this group and elsewhere on the Internet) hasn't turned up more than a few threads. I did see Pat mention in one thread that if a model's .rb file failed to compile for some reason, TS would be unable to parse it so would skip it, so I tested one of the skipped models by firing up "rails console" and just typing the name of that model to see if any errors came back... but I saw no errors.
>
> I also saw a thread where Pat suggested a workaround of using an "indexed_models" block in config/sphinx.yml, but doing that (listing all models that should be indexed), and then deleting development.sphinx.conf and all of Sphinx's database files, then running "rake thinking_sphinx:index" just gets me this output:
> Generating Configuration to /Users/arussell/Sites/deucescracked-rails3/config/development.sphinx.conf
> rake aborted!
> You have a nil object when you didn't expect it!
> You might have expected an instance of Array.
> The error occurred while evaluating nil.select
>
> (See full trace by running task with --trace)
>
> Possibly relevant information:
> • Mac OS X Lion (10.7.0)
> • Ruby 1.8.7 (Ruby Enterprise Edition 2011.03)
> • Sphinx v0.9.9 (r2117)
> •
> • Relevant gems:
> •
> • riddle (1.4.0)
> • thinking-sphinx (2.0.5)
> •
> • ts-datetime-delta (1.0.2)
> • ts-delayed-delta (1.1.2)
> Does anyone have any other suggestions I could try?
>
> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/PqUpbl-H9ooJ.
> To post to this group, send email to thinkin...@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.

[Aaron B. Russell]

(Resubmitting OP as a reply because I need to sanitise it and it doesn't seem like I can edit Google Groups posts, apologies for post order confusion to anyone reading, this came before Pat's response above...)

I recently wiped my laptop to install Mac OS X Lion, and am having some trouble setting up Thinking Sphinx in my development environment again. This situation has been replicated on another developer's laptop while he was trying to set up the same project.

When trying to run "rake thinking_sphinx:index" for the first time and letting it autocreate my development.sphinx.conf, I'm seeing certain models being completely skipped. Not mentioned in the output of "rake thinking_sphinx:index", and no mention of them in the resulting development.sphinx.conf ... it's as if I've failed to mention that those models should be indexed. These models were previously being indexed... infact they still are in our production environment, but of course we don't want to remove the production.sphinx.conf there and try rebuilding because it seems quite likely that our production environment will just break the same way it has on our development machines.

Searching for people having similar problems (both here in this group and elsewhere on the Internet) hasn't turned up more than a few threads. I did see Pat mention in one thread that if a model's .rb file failed to compile for some reason, TS would be unable to parse it so would skip it, so I tested one of the skipped models by firing up "rails console" and just typing the name of that model to see if any errors came back... but I saw no errors.

I also saw a thread where Pat suggested a workaround of using an "indexed_models" block in config/sphinx.yml, but doing that (listing all models that should be indexed), and then deleting development.sphinx.conf and all of Sphinx's database files, then running "rake thinking_sphinx:index" just gets me this output:

Generating Configuration to /Users/arussell/Sites/projectname/config/development.sphinx.conf

rake aborted!

You have a nil object when you didn't expect it!

You might have expected an instance of Array.

The error occurred while evaluating nil.select

(See full trace by running task with --trace)

[Aaron B. Russell]

Hi Pat, thanks for getting back to me. It's probably not Lion specific but figured I should mention that anyway. It's probably just affecting me all of a sudden because I've started from scratch and blew away my previous development.sphinx.conf.

Here's my current sphinx.yml:

defaults: &defaults

  match_mode: extended2

  enable_star: 1

  min_prefix_len: 3

  auto_escape_characters: |-!@*<()

  bin_path: /usr/local/bin/

  searchd_binary_name: searchd

development:

  mem_limit: 256m

  searchd_file_path: /Users/arussell/Sites/projectname/db/sphinx

  searchd_files: /Users/arussell/Sites/projectname/db/sphinx

  config_file: /Users/arussell/Sites/projectname/config/development.sphinx.conf

  indexed_models:

    - Article

    - Blog

    - BlogComment

    - BlogEntry

    - Coach

    - Forum

    - ForumCategory

    - Playlist

    - Podcast

    - Podshow

    - Post

    - Series

    - Topic

    - User

    - Video

    - VideoFile

  <<: *defaults

[Pat Allan]

Hi Aaron

The syntax looks fine to me - can you provide the entire output when running the following:
rake ts:configure --trace

--
Pat

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/wdurhcPi_hUJ.

[Aaron B. Russell]

Hi Pat,

With the indexed_models statement in sphinx.yml, I get this output:

piro:projectname arussell$ rake thinking_sphinx:configure --trace

(in /Users/arussell/Sites/projectname)

** Invoke thinking_sphinx:configure (first_time)

** Invoke thinking_sphinx:app_env (first_time)

** Execute thinking_sphinx:app_env

=> Debugger enabled

** Execute thinking_sphinx:configure

Generating Configuration to /Users/arussell/Sites/projectname/config/development.sphinx.conf

rake aborted!

You have a nil object when you didn't expect it!

You might have expected an instance of Array.

The error occurred while evaluating nil.select

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@dc3/bundler/gems/thinking-sphinx-34510f41a2c4/lib/thinking_sphinx/active_record.rb:309:in `local_sphinx_indexes'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@dc3/bundler/gems/thinking-sphinx-34510f41a2c4/lib/thinking_sphinx/active_record.rb:244:in `to_riddle'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@dc3/bundler/gems/thinking-sphinx-34510f41a2c4/lib/thinking_sphinx/configuration.rb:146:in `generate'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@dc3/bundler/gems/thinking-sphinx-34510f41a2c4/lib/thinking_sphinx/configuration.rb:143:in `each'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@dc3/bundler/gems/thinking-sphinx-34510f41a2c4/lib/thinking_sphinx/configuration.rb:143:in `generate'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@dc3/bundler/gems/thinking-sphinx-34510f41a2c4/lib/thinking_sphinx/configuration.rb:159:in `build'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@dc3/bundler/gems/thinking-sphinx-34510f41a2c4/lib/thinking_sphinx/tasks.rb:70

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:636:in `call'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:636:in `execute'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:631:in `each'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:631:in `execute'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:597:in `invoke_with_call_chain'

/Users/arussell/.rvm/rubies/ree-1.8.7-2011.03/lib/ruby/1.8/monitor.rb:242:in `synchronize'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:2051:in `invoke_task'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:2029:in `each'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:2023:in `top_level'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:2001:in `run'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/lib/rake.rb:1998:in `run'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/gems/rake-0.8.7/bin/rake:31

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/bin/rake:19:in `load'

/Users/arussell/.rvm/gems/ree-1.8.7-2011.03@global/bin/rake:19

Without the indexed_models section in sphinx.yml I see the same thing, but the last line is "Generating configuration..." without the errors (though some of my models are missing from development.sphinx.conf).

[Pat Allan]

That's odd - the object it's saying is nil is a class-level array (sphinx_indexes) on the model, which is set when define_index is called...

You're not manually requiring the models anywhere, are you?

--
Pat

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/_9TSgepQD7kJ.

[Aaron B. Russell]

Hrm, it does seem that some of the models are indeed manually requiring other models. At first glance, I'm not sure why we're doing that though. From the sound of your last message it seems like requiring models from other models is a bad thing to do?

[Pat Allan]

It's more that Rails doesn't like it - because it has its own autoloading mechanisms, and so it can end up double-loading classes, which can have odd side effects.

If you can, try removing all the manual requires (at least ones related to models), and see if that helps things - both with and without the manual indexed_models setting in sphinx.yml.

--
Pat

On 13/08/2011, at 11:27 PM, Aaron B. Russell wrote:

> Hrm, it does seem that some of the models are indeed manually requiring other models. At first glance, I'm not sure why we're doing that though. From the sound of your last message it seems like requiring models from other models is a bad thing to do?
>

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/6qsSyLqpmLQJ.

[Aaron B. Russell]

Thanks for explaining. :)

Okay so digging a little deeper I found that it was just a custom class with no superclass (so not actually a model) that just happened to be in app/models (the file's location threw me). Moving it to lib/ made no difference unfortunately, I'm getting the exact same results as before.

[Pat Allan]

So there's no manual require statements any more?

--
Pat

On 15/08/2011, at 8:13 AM, Aaron B. Russell wrote:

> Thanks for explaining. :)
>
> Okay so digging a little deeper I found that it was just a custom class with no superclass (so not actually a model) that just happened to be in app/models (the file's location threw me). Moving it to lib/ made no difference unfortunately, I'm getting the exact same results as before.
>

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/E963TOSBAlgJ.

[Aaron B. Russell]

No models are being manually required anywhere in the project. Some of the models do have require statements to load in things from outside the app that they need to run, but that's it.