ThinkingSphinx.indexed_models Problem

[karmacoma]

Hi,
Using the latest version of thinking-sphinx (1.3.6). I am getting the
following error: "undefined method `name' for nil:NilClass" when
destroying a record.

The stack trace leads me to the following file: thinking-sphinx/lib/
thinking_sphinx/active_record.rb:229

def eldest_indexed_ancestor
ancestors.reverse.detect { |ancestor|
ThinkingSphinx.indexed_models.include?(ancestor.name)
}.name
end

When the destroy method is called, "ThinkingSphinx.indexed_models"
returns an empty array.

However, according to my application it should be returning:
["Administrator", "Article", "Customer", "Order", "Product",
"Representative"].

* Within production it raises the exception every time (obviously, as
the classes are cached).
* Within development it raises this exception on the first invocation,
but works on the second attempt, and subsequent attempts.

This seems to be an issue related to lazy loading?

In my application I often have to delete records. So I am hoping
someone can help me fix this problem.

Kind regards, Oliver Beddows.

[Pat Allan]

Hi Oliver

I'm kinda surprised by this - I thought that array was being populated
at the beginning of each web request in the dev environment, but just
so I can track it down... what version of Rails are you using?

--
Pat

> --
>
> You received this message because you are subscribed to the Google
> Groups "Thinking Sphinx" group.
> To post to this group, send email to thinkin...@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com
> .
> For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en
> .
>
>

[karmacoma]

Hi Pat,

I am using rails version 2.3.4.

Starting a debugger just before: thinking-sphinx/lib/thinking_sphinx/
active_record.rb:83
I can see the list of indexed models being gathered as script/server
boots up.

Then if I place a breakpoint just before: vendor/plugins/thinking-
sphinx/lib/thinking_sphinx.rb:63
Upon hitting the first request, Thread.current
[:thinking_sphinx_indexed_models] returns nil.

I am using the following version of ruby in development:
- ruby 1.8.7 (2009-06-12 patchlevel 174) [i686-darwin10]

And in production:
- ruby 1.8.6 (2007-09-24 patchlevel 111) [i486-linux]

Hope this helps...

Kind regards, Oliver Beddows.

[Pat Allan]

Hi Oliver

Can you give the latest release (1.3.7) a shot? It should fix this
problem.

--
Pat

[karmacoma]

Hi Pat,

Okay, deleting a record now works (which is great!).
However, now I suffering a slightly different problem.

If I perform a search; Destroy one of the records within the results;
Then re-perform the search.
I now get the following error:

===
"You have a nil object when you didn't expect it!
The error occurred while evaluating nil.local_options"
===

With the stack trace leading me to: thinking-sphinx/lib/
thinking_sphinx/search.rb:310:in `client'.

Secondly, I am unable to re-index my records, as I get the following
error when I issue rake ts:index:

===
Generating Configuration to /Users/Oliver/git/greyville/config/
development.sphinx.conf
Sphinx 0.9.8.1-release (r1533)
Copyright (c) 2001-2008, Andrew Aksyonoff

using config file '/Users/Oliver/git/greyville/config/
development.sphinx.conf'...
ERROR: section 'order_core_0' (type='source') already exists in /Users/
Oliver/git/greyville/config/development.sphinx.conf line 75 col 1.
===

Kind regards, Oliver.

[Pat Allan]

Okay, I just re-issued the same gem, and updated GitHub, to fix the
first of these two new issues. Still investigating the second...

--
Pat

On 28/11/2009, at 11:23 PM, karmacoma wrote:

> Hi Pat,
>

[Pat Allan]

Hmm, I can't seem to reproduce the indexing issue...

How do you have TS installed? (gem or plugin) And what's your index
definition looking like in Order?

--
Pat

On 28/11/2009, at 11:23 PM, karmacoma wrote:

> Hi Pat,
>

[karmacoma]

Hi,
I have thinking-sphinx installed as a vendor/plugin.

My index definition for looks like:

===
define_index do
# Fields
#
indexes :id, :as => :order_id
indexes customer.account_ref
indexes customer.contact_name, customer.company_name, :as
=> :customer_name
indexes line_items.code
indexes line_items.name

# Attributes
#
has :status_code

# Properties
#
set_property :delta => true
end
===

Here is what I have be able to establish so far.
Note: I'm testing all of this in development mode.

-> With config.cache_classes = true

1. If I perform a search, destroy a record within the results, re-
perform the same search.
I still get the same "nil.local_options" error as described in my
previous email.

2. I am unable to rotate my indexes using rake ts:index.

-> With config.cache_classes = false

1. If I perform a search, destroy a record within the results, re-
perform the same search.
I get a RuntimeError: "Called id for nil, which would mistakenly be 4"

2. I am able to rotate my indexes.

Thanks for your efforts.
Kind regards, Oliver.

[Pat Allan]

Okay, there's been another update...

Current status:

-> With config.cache_classes = true

1. Fixed
2. Can't reproduce

-> With config.cache_classes = false

1. Can't reproduce
2. All fine.

It's possible that in fixing the nil.local_options issue, I've fixed
others. If you could give it a spin, that'd be great.

Also, one thing you might want to change in your index definition:

indexes customer.contact_name, customer.company_name, :as
=> :customer_name

To:
indexes [customer.contact_name, customer.company_name], :as
=> :customer_name

You were creating two fields with the same name, and I'm not sure how
Sphinx handles that. My change combines the two columns into a single
field called customer_name.

--
Pat

[karmacoma]

Hi Pat,
Many thanks! I have just tried out your most recent commit, and it
appears to have fixed my main problem of destroying records within the
index.

Thanks again for the pointer on my index definition; I hadn't noticed
that.

However, I am still trying to resolve the issue of rebuilding my
indexes in development mode, with cache_classes set to true.
If I disable these two lines in my Order index definition, everything
works fine:

indexes customer.account_ref

indexes [customer.contact_name, customer.company_name], :as
=> :customer_name

I also have an index definition on my Customer model using these same
fields: account_ref, contact_name, customer_name.
Disabling these fields within the Customer index definition has no
effect either way.

The only way I can get it to work is by disabling these two field
indexes.
The problem is that my application relies upon them, and I'm reluctant
to push any changes to production that prevent me from rotating my
indexes.

Kind regards, Oliver.

[karmacoma]

Correction:

I also have an index definition on my Customer model using these same

fields: account_ref, contact_name, *company_name*.

[Pat Allan]

Hi Oliver

Unfortunately, I can't reproduce this - I created a test app with
order and company models, with a belongs_to association, and indexes
on both, including the delta index on order. All indexes fine.

If you'd like, perhaps you can send me a copy of your app, and I can
try to make the issue happen on my machine?

--
Pat

[karmacoma]

Hello Pat,

I have been able to reproduce the problem by creating a new rails
application
from scratch with thinking-sphinx installed as a vendor plugin.

If I do the following:
* Execute "rails test_app"
* Configure the development environment to run with cache_classes set
to true (to mimic production)
* Create the migrations and define models with indexes (as listed
below)
* Generate an Observer on the Order model: "ruby script/generate
observer Order"
* Activate the OrderObserver within environment.rb
* Execute: "rake ts:config && rake ts:index"

I get the same error that I've been harping on about ;o)
As soon as I disable the observer, and re-configure thinking-sphinx; I
am able to rotate my indexes correctly.

So it appears to be related to my use of an Observer on the Order
model.
Note: The observer is "as generated" without any additional code.

I hope this clarifies the problem I have been having.
Kind regards, Oliver.

======
Models:
======

class Order < ActiveRecord::Base
belongs_to :customer

define_index do
indexes customer.account_ref, :as => :account_ref

indexes [customer.contact_name, customer.company_name], :as
=> :customer_name

end
end

class Customer < ActiveRecord::Base
has_many :orders

define_index do
indexes :account_ref
indexes :contact_name
indexes :company_name
end
end

========
Migrations:
========

class CreateCustomers < ActiveRecord::Migration
def self.up
create_table :customers do |t|
t.account_ref, :string
t.contact_name :string
t.company_name :string
t.timestamps
end
end

def self.down
drop_table :customers
end
end

class CreateOrders < ActiveRecord::Migration
def self.up
create_table :orders do |t|
t.customer_id :integer
t.timestamps
end
end

def self.down
drop_table :orders
end
end

> ...
>
> read more »

[Pat Allan]

Right, now I can reproduce it too - thanks for that process, it's
definitely activating the observer that causes the problem. I'll see
if I can figure out the source today.

Cheers

--
Pat

[Pat Allan]

Figured it out!

If anyone cares about the internals... the observer changed the load
procedure with cached classes, looking at Order first, instead of
loading all models... so Thinking Sphinx was presuming that all
classes had been loaded, and so it loops through the known indexed
models.

So, at this point, because of the observer, that array was ["Order"],
instead of ["Company", "Order"] (enforced alphabetical order - this is
key). When Order is processed, suddenly Rails then knows about Company
because of the associations, and so the array becomes ["Company",
"Order"]. So the loop runs through again with the second item - which
is Order, again!

Just pushed a patch that fixes this - now Thinking Sphinx will
*always* try to load all models, even if classes are cached. If they
are, nothing happens. If they're not, or if observers are getting in
the way, then we ensure that all indexed models are in that array from
the start.

So, installing TS again as a plugin should fix this issue... gem bump
will happen later today.

--
Pat

>> To post to this group, send email to thinking-
>> sph...@googlegroups.com.

[karmacoma]

Hi Pat,

Excellent! Many thanks for sorting this out :o)
I really appreciate it...

Regards, Oliver.

> ...
>
> read more »