a bunch of (maybe) related issues: delta indexes, using sphinx w/testing, and b9e4294028a541fc58f6a5829db70be30b85fa19

[agib]

I'm probably confusing myself, but I'll try to explain...

I ran into this issue when trying to use sphinx with my cucumber
feature (tests). I've decided to enable TS for my cucumber tests by
specifying:

ThinkingSphinx.updates_enabled = true
ThinkingSphinx.deltas_enabled = true

in my features/steps/env.rb file.

I also have the following in my config/sphinx.yml:

development:
port: 3312

test:
port: 3313

So basically I run two indexers at once when I'm working
(RAILS_ENV=development rake ts:start && RAILS_ENV=test rake ts:start).

I find that when I use the HEAD of the master branch
(b9e4294028a541fc58f6a5829db70be30b85fa19 as of this message) I get
the array of nil problem:

class Context < AR::Base
# ...
define_index do
indexes :name, :sortable => true
indexes :desc

has state_status, created_at
set_property :delta => true
end
# ...
end

> Factory.create(:context, :name => 'context 1')
> Factory.create(:context, :name => 'context 2')
> Factory.create(:context, :name => 'context 3')

...

using config file '/Users/agib/Sites/site/config/test.sphinx.conf'...
indexing index 'context_delta'...
collected 1 docs, 0.0 MB
sorted 0.0 Mhits, 100.0% done
total 1 docs, 27 bytes
total 0.015 sec, 1793.07 bytes/sec, 66.41 docs/sec
rotating indices: succesfully sent SIGHUP to searchd (pid=3572).
Sphinx 0.9.8.1-release (r1533)
Copyright (c) 2001-2008, Andrew Aksyonoff

...

> Context.search 'context'

=> [nil, nil, nil]

I switched back to 3e5fc7370b93e543ffd6ca2e0f240bec47baf4b9 and no
longer have the problem... has anyone else had similar issues?

Thanks in advance for any help!

Best,
Aaron

[Pat Allan]

Hi Aaron

To be honest, I'm not quite sure where to start on this issue - but
then, it is late (but it needs a response anyway, right?)

Are you able to narrow down the commit where it stops working at all?
I realise that's a bit fiddly (although if you have it as a live git
repo, you can checkout specific commits, not just branches (something
I've learnt recently - apologies if this is old news).

--
Pat

[agib]

Hi Pat,

Thank you for your reply!

I'm sorry, I kind of lost track of what I was doing there... I think
the problem may have come out of my hacky attempt to get TS running
with my cucumber integration tests. Have you figured out any good ways
to do integration testing with TS? i.e. is there a way to call rake
ts:index from the test code?

For example, is there a good way to implement the "Given TS has been
indexed" step below in the 3rd example? I also included some examples
to illustrate my understanding of delta indexing -- are they correct?
---------------------------
class Context < ActiveRecord::Base

define_index do
indexes :name, :sortable => true

has state_status, created_at
set_property :delta => true
end

end
---------------------------
Scenario: #1 searching for contexts (no indexing)
Given the following contexts exist:
| name |
| context 1 |
| context 2 |
| context 3 |
And TS has NOT been indexed # no-op
When I go to '/contexts?search=context' # something like:
Context.search(params[:search], :order => params[:order_by])
Then I should see 'context 1'
And I should see 'context 2'
And I should see 'context 3'

Scenario: #2 searching for contexts arranged by name (no indexing)
Given the following contexts exist:
| name |
| context 1 |
| context 2 |
| context 3 |
And TS has NOT been indexed # no-op
When I go to '/contexts?search=context&order_by=name'
Then I should see no contexts

Scenario: #3 searching for contexts arranged by created_at (no
indexing)
Given the following contexts exist:
| name |
| context 1 |
| context 2 |
| context 3 |
And TS has NOT been indexed # no-op
When I go to '/contexts?search=context&order_by=created_at'
Then I should see 'context 1'
And I should see 'context 2'
And I should see 'context 3'

Scenario: #4 searching for contexts arranged by name (after indexing)
Given the following contexts exist:
| name |
| context 1 |
| context 2 |
| context 3 |
And TS has been indexed
When I go to '/contexts?search=context&order_by=name'
Then I should see 'context 1'
And I should see 'context 2'
And I should see 'context 3'

In Scenario #2, nothing should show up, right? Could I create a string-
to-int style attribute to allow for this to work? (since sorted
integer attributes are stored in delta indexes, right?)

Sorry for all the questions!!! And thank you in advance for any help!

Best,
Aaron

P.S. How was Cambodia? I visited Siem Reap last year and loved it.

[Pat Allan]

Hmm, there isn't yet a nice neat way to control Sphinx *within*
tests... you can run the rake tasks normally before running tests, but
obviously that's not so much help when you need to test certain parts
of the code as data changes.

So, the quick and dirty way, is to invoke the rake tasks within code.

require 'rake'
require 'rake/testtask'
require 'rake/rdoctask'
require 'tasks/rails'

# ...

Rake::Task['ts:in'].invoke

I've no idea what environment it uses though, but hopefully it runs
within the current scope's environment.

As for sorting, items will still be returned - unless they've been
created since your last full index, and you're not using delta
indexes. The order will not reflect any changes though, and the string-
to-int algorithm is applied to each index separately (so whatever's
first in the delta index will appear next to whatever's first in the
core index, etc).

Hope this helps until I can get a better solution sorted.

--
Pat

[agib]

Hey Pat, thank you for the help! I'll give it a try and see how it
works out.

Best,
Aaron