Problem (and possible bug?) with global search

[Timo Virkkala]

Hi,

I found a problem (and a hackaround solution) with the global search in Thinking Sphinx.

When I try to use ThinkingSphinx.search(term), I get a traceback like the following:

NoMethodError: undefined method `constantize' for 0:Fixnum

        from lib/thinking_sphinx/search.rb:936:in `class_from_crc'

        from lib/thinking_sphinx/search.rb:915:in `instances_from_matches'

        from lib/active_support/ordered_hash.rb:139:in `each'

        from lib/active_support/ordered_hash.rb:139:in `each'

        from lib/thinking_sphinx/search.rb:913:in `instances_from_matches'

        from lib/thinking_sphinx/search.rb:436:in `compose_results'

        [etc.]

(I cleaned up the pathnames a bit)

My (relevant) environment:

  * rails (3.0.9)

  * riddle (1.4.0)

  * thinking-sphinx (2.0.5 dc98d80)

  * sphinx (2.0.1-beta)

I've managed to locate the problem: there are a couple of Sphinx version number comparisons in ThinkingSphinx::Search, which choose what to do based on the Sphinx version number. For some reason the correct actions would be the ones chosen for Sphinx versions < 2, even though I'm running Sphinx 2.0.1.

I can get the search to run correctly with the following monkey-patch:

class ThinkingSphinx::Search

 def class_from_crc(crc)

  # if Riddle.loaded_version.to_i < 2

   config.models_by_crc[crc].constantize

  # else

  #  crc.constantize

  # end

 end

 def crc_attribute

  # Riddle.loaded_version.to_i < 2 ? 'class_crc' : 'sphinx_internal_class'

  'class_crc'

 end

end

Does anybody have an idea of why this is? Does Thinking Sphinx make incorrect assumptions about the column names returned by sphinx, or do I have a configuration error somewhere?

- Timo Virkkala

[Pat Allan]

Hi Timo

You're not the only person to hit this - someone's logged a similar issue on Github. To be honest, I'm a little surprised this is happening - as Sphinx 2.0.x should be storing the class name as a string attribute (more reliable than the old CRC approach, where collisions could potentially happen).

Is this only happening in development? Or production? Is the model in question using STI and/or is there a column called 'type'?

--
Pat

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/g5JBaJp8TV4J.
> To post to this group, send email to thinkin...@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.

[Timo Virkkala]

Hi Pat,

And thanks for the quick reply. Yeah, it happens in all environments. We do have some STI with a column called 'type' in one table. But the problem exists even when I limit indexing to only one of our models (and remove the older index files), and that one doesn't have a column called 'type'.

-Timo

[Pat Allan]

Hi Timo

Thanks for getting back to me. Can you share your development.sphinx.conf file in a gist or similar? Make sure you remove the database password from each source.

--
Pat

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/2ebUP3XjggEJ.

[Timo Virkkala]

Hi,

I'll try to create a minimal example and post that somewhere.
Meanwhile, some snippets from the current conf file:

source brand_core_0
{
[...]
sql_query = SELECT [...] 3717453404 AS `class_crc`, IFNULL('Brand',
'') AS `sphinx_internal_class` [...]
[...]
sql_attr_uint = class_crc
sql_attr_str2ordinal = sphinx_internal_class
[...]
}

Is that what you were looking for? That's from a table which does not
have STI. One STI table has the following:

source party_core_0
{
[...]
sql_query = SELECT [...] CASE IFNULL(`parties`.`type`, '') WHEN
'Person' THEN 863032384 WHEN 'Organization' THEN 3655317636 ELSE
1213489636 END AS `class_crc`, IFNULL(`parties`.`type`, '') AS
`sphinx_internal_class` [...]
[...]
sql_attr_uint = class_crc
sql_attr_str2ordinal = sphinx_internal_class
[...]
}

-Timo

[Pat Allan]

Right - the sphinx_internal_class shouldn't be sql_attr_str2ordinal, but sql_attr_string. That's the problem, but as for the cause... can you try the latest TS gem release instead of Github? v2.0.7

--
Pat

[Timo Virkkala]

Hi,

The problem persists with 2.0.7.

I investigated further. The current monkey-patch I need to get things
working is:

class ThinkingSphinx::Attribute
SphinxTypeMappings[:string] = :sql_attr_string
end

So, it seems that setting that in ThinkingSphinx::Attribute somehow
fails. Normally the code in ThinkingSphinx::Attribute checks
Riddle.loaded_version.to_i > 1 and sets that if the result is true.

Could it be that Riddle.loaded_version fails at that point?

-Timo

[Pat Allan]

Looks like it - can you check the value of Riddle.loaded_version in script/console?

--
Pat

[Timo Virkkala]

Loading development environment (Rails 3.0.9)
irb(main):001:0> Riddle.loaded_version
=> "2.0.1"
irb(main):002:0> Riddle.loaded_version.to_i
=> 2

No problems there. I'm just wondering - Riddle seems to check the
loaded version by running the indexer binary. But at what time does it
load the bin_path from the configuration? Could it be that at the time
the SphinxTypeMappings map is built, the bin_path is not yet known? Or
does it maybe run the indexer at that point with a wrong path?

-Timo

[Pat Allan]

Ah, I think you've got it - the bin_path setting hasn't come into play when ThinkingSphinx::Attribute is defined, and I'm pretty sure that's the problem.

I'll try to get a fix out in the next 24 hours - but your workaround looks pretty good in the meantime.

Thanks for the help Timo :)

--
Pat

[Pat Allan]

Okay, this is now fixed in the latest commits to both branches, if anyone wants to give it a spin.

--
Pat

[Timo Virkkala]

Yay! It works now without any monkey-patching.

Many thanks for your support, Pat!

-Timo