problem with sphinx returning no results but not returning "0" results

[agibralter]

I'm not too sure what to make of this... has anyone else ever had
trouble with sphinx not explicitly returning "0" results but rather
just returning nil? I can't quite figure out why sphinx is doing it,
but it seems like TS might still want to check for nil in its
total_pages method because it throws undefined method for nil
exceptions.

https://gist.github.com/e67cd724cf4c58f8f4ed

[Pat Allan]

Hi Aaron

I think this only happens when Sphinx throws a warning or error,
instead of actually returning 0 results. What's the query you're
running? (And yes, it should be patched anyway - I think there might
be a fork somewhere that fixes this).

--
Pat

> --
>
> You received this message because you are subscribed to the Google
> Groups "Thinking Sphinx" group.
> To post to this group, send email to thinkin...@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com
> .
> For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en
> .
>
>

[agibralter]

Hi Pat,

Thanks for the response -- so this is really weird: I have the same
ruby/rails code running on dev and production right now, but my dev
box is OS X and production is CentOS. I even have the same source of
Sphinx (Sphinx 0.9.9-rc2 (r1785)) compiled on both...

The following query causes my production machine to raise the no
method exception (due to my total_pages call) but works just fine on
my dev box: "Would it be funny to see ZA NEWS tv commercials on SABC
(when they were not going to run the show)?" As I put in the gist, I
first escape that query, split it, and join it with ORs: query.gsub
(Riddle.escape_pattern, ' ').split(/\s+/).join(' | '). This gives me
the following raw query for sphinx: 'Would | it | be | funny | to |
see | ZA | NEWS | tv | commercials | on | SABC | when | they | were |
not | going | to | run | the | show | ?' (I do this so that the query
is run as a pure BM25 search -- I want the rare words to rank higher,
regardless of phrase construction).

Any thoughts on how to figure out what is causing the Sphinx error?

As for TS, I think I'll just rescue until the patch fixes the
total_pages issue...

Thanks again,
Aaron

[Pat Allan]

Hi Aaron

On your production machine, is there anything in the error log of your
web server? (assuming you're using passenger - the apache/nginx logs
are where any 'puts' results end up). If there isn't, then I've no
idea - but I should have this patched some time today.

Cheers

--
Pat

[agibralter]

Hmm, well the errors that came through on Hoptoad from production were
simply NoMethodErrors thrown because total_pages was trying to divide
nil: @total_pages ||= (@results[:total] / per_page.to_f).ceil =>
@results[:total] was nil I guess. Nothing was in there about Sphinx
though. I think Riddle keeps the Sphinx errors pretty silent, no?

This is really strange and I can't seem to recreate any of the
problems consistently... I was at one point getting the same issues on
my dev box in the console. You know how the log shows something like:

Sphinx Querying: 'Would | it | be | funny | to | see | ZA | NEWS |

tv | commercials | on | SABC | (when | they | were | not | going | to

| run | the | show)?'
Sphinx (0.011262s) Found 0 results

Well it said:

Sphinx Querying: 'Would | it | be | funny | to | see | ZA | NEWS |

tv | commercials | on | SABC | (when | they | were | not | going | to

| run | the | show)?'
Sphinx (0.011262s) Found results

I will continue to try to recreate the error... I will also look
through sphinx.query.log to see if anything pops up.

[agibralter]

Also, is there a preferred method of searching just words? What I'm
trying to achieve is something where people type in a question in
plain english and the app returns similar questions that other people
already asked (like GetSatisfaction I think). Sometimes people might
search something like: "Are you happy/excited about blah blah?" I
don't think escaping the slash there makes sense... rather I think non-
text characters should be replaced with spaces... Does query.gsub
(Riddle.escape_pattern, ' ') make sense for this? Or should I do a
gsub with /[^\w]/ ?

Also, I split on space and join with ORs because I want the rarest
(and most likely *key*) words to be ranked the highest using the bm25
ranking algorithm.

[Pat Allan]

I would just replace non-word chars with spaces...

gsub(/\W+/, ' ') should do the trick

--
Pat

[agibralter]

Worked great! Thank you!