How rebuild the indexes during the testing?

[mix]

Hi, i test all my application with unit and integration tests. How can
i rebuild automatically the indexes at the start of every test (maybe
just the one i choose, to avoid waste of time for an unused reindex).
Actually when I do a Model.search the results are 0, also if in the
fixtures there are some. With ferret i used Model.rebuild_index at the
start of the method. Is there something with sphinx too? How do you do
the testing?

[gobigdave]

I avoid the issue by using flexmock to verify that search is called
with the expected parameters and do not test that search returns
expected values since thinking_sphinx and sphinx itself are tested
separately. After all, you are testing your models and controllers,
not sphinx. For example:

q = 'pdi'
flexmock(Post).should_receive(:search).once.with(q).and_return do
posts = []
WillPaginate::Collection.create(1, posts.size > 0 ? posts.size :
Post.per_page, posts.size) do |pager|
pager.replace(posts)
pager.total_entries = posts.size unless pager.total_entries
end
end

[mix]

Unfortunately i've to test also the object returned because i'm using
a method which does some logic stuff and then call the sphinx search,
and i'd be a lot better testing the results too (having them ordered
by a specific column i haven't the risk of the change of the sphinx
ranking algorithm).
Do you know if (and how) to test results too? With the possibility to
rebuild the index at the begin of a unit test (would the delta indexes
solve this?).

(@Pat, if you're reading it would be great an answer from you too :) )

[Pat Allan]

At this point in time, I can offer two options.. the first is some
rspec matchers:
http://openmonkey.com/articles/2009/07/thinking-sphinx-rspec-matchers

The second is a starting point for cucumber:
# from env.rb -- make sure you disable transactional fixtures, as well.
ThinkingSphinx.deltas_enabled = true
ThinkingSphinx.updates_enabled = true
ThinkingSphinx.suppress_delta_output = true

FileUtils.mkdir_p
ThinkingSphinx::Configuration.instance.searchd_file_path

ThinkingSphinx::Configuration.instance.build
ThinkingSphinx::Configuration.instance.controller.index
ThinkingSphinx::Configuration.instance.controller.start

at_exit do
ThinkingSphinx::Configuration.instance.controller.stop
end

# from common_steps.rb
Before do
# Because we're not using transactions (Sphinx needs data in tables)
%w( table names go here ).each do |table|
ActiveRecord::Base.connection.execute "TRUNCATE TABLE #{table}"
end

# Let's not store emails between scenarios
ActionMailer::Base.deliveries.clear
end

Given 'the Sphinx indexes are updated' do
ThinkingSphinx::Configuration.instance.controller.index
sleep(0.25)
end


It's still far from perfect, but it's a start.

--
Pat

[mix]

Hi Pat, thank you for the answer.
Actually i'm using just the ruby/rails core testing (not rspec or
cucumber or something else).
So i've added to my env.rb (using an initializer):

ThinkingSphinx.deltas_enabled = true
ThinkingSphinx.updates_enabled = true
ThinkingSphinx.suppress_delta_output = true

Then in my model_test.rb i've written a simple test:

def test_simple_test

ThinkingSphinx::Configuration.instance.build
ThinkingSphinx::Configuration.instance.controller.index
ThinkingSphinx::Configuration.instance.controller.start

results = Model.full_text_search('')
assert_equal 2, results.total_entries
assert_equal models(:second), results[0]
assert_equal models(:first), results[1]
ThinkingSphinx::Configuration.instance.controller.stop
end

And it's working! :)

The only problem i've got is when i have something like this:

def test_simple_test
assert_equal 'abc', models(:second).detail.tag
models(:second).detail.update_attribute(:tag, 'abc')

ThinkingSphinx::Configuration.instance.build
ThinkingSphinx::Configuration.instance.controller.index
ThinkingSphinx::Configuration.instance.controller.start

results = Model.full_text_search(:conditions => {:tags => 'cde')
assert_equal 0, results.total_entries
ThinkingSphinx::Configuration.instance.controller.stop
end

This is the correct test, but i get one result: the :second item,
which i updated the tag value to another value. It seems that it
doesn't get the new value when sphinx reindex. I've tried to puts the
tag facets and instead of 'abc' there is nil... any idea of the
possible cause?

[Pat Allan]

What's the definition of the Model.full_text_search method?

--
Pat

[mix]

i forgot to paste it and i did some mess :)
it's a method which does some stuff before calling the sphinx search.
Btw i've also tried directly calling sphinx search (to see if my
method was broken) and you can try this:

A much clearer example:

In the db is this structure:

article -> tags

an index collect the tag's name using:
indexes self.tags(:name), :as => :tags

def test_simple_test
assert_equal 1, articles(:second).tags.size # <-- this article has
just one tag
assert_equal tags(:cde), articles(:second).tags.first # <-- there
is only this object associated to this tag
# assert_equal 1, tags(:cde).articles.size
assert_equal 'cde', tags(:cde).name
tags(:cde).update_attribute(:name, 'abc')
# here i've also tried with a tags(:cde).reload

ThinkingSphinx::Configuration.instance.build
ThinkingSphinx::Configuration.instance.controller.index
ThinkingSphinx::Configuration.instance.controller.start

results = Article.search('', :conditions => {:tags => 'cde')
assert_equal 0, results.total_entries # <-- this is what it should
be, but there is one result... the articles(:second), which has still
associated the old tag.
# you can for example do a puts results.first.tags.map(&:name) to
see that the tag has the name changed, but sphinx get it anyway
ThinkingSphinx::Configuration.instance.controller.stop
end

I've tried all this also in the dev mode using the console and rebuild
the index with ts:in, and all works correctly, so it's just a matter
of the test mode i think. Should i update in same way the tags table
during the test? (how?)

[Pat Allan]

I'm afraid I've hit a wall with this issue. Seems like you're doing
everything right, and especially since delta indexes don't come into
play, I'm confused as to why results are being returned. Can you look
at the database as part of the test, check that the data in there is
correct?

Better yet (if the data isn't blown away after the test), check your
database manually, to ensure the data's there outside of Rails
transactions.

--
Pat