exclude query on xapian backend

[s|s]

Hi,
Just ran few queries on my index which were just exclude type. It ends
up with an error <expression> NOT <expression>. It seems one needs
FLAG_PURE_NOT http://xapian.org/docs/apidoc/html/classXapian_1_1QueryParser.html#fdaec563f70612cc1c6939226ebdb1e484cf1fd3c0289633db308d687bed8cf2
to run "NOT <something>" type query.

Patch included

--
~preet~

[David Sauve]

Thanks, Supreet.  I've applied the patch and pushed to GitHub.  I've also added tests to ensure it doesn't happen again.

[notanumber]

Just an update on this issue. FLAG_PURE_NOT should only be set when
the query string consists of only a NOT expression and it can consume
quite a bit of memory since it has to load all documents to do so.
I've updated the code to only do this when needed. Let me know if
it's not behaving as expected.

David

On Jul 30, 8:57 am, David Sauve <dnsa...@gmail.com> wrote:
> Thanks, Supreet.  I've applied the patch and pushed to GitHub.  I've also
> added tests to ensure it doesn't happen again.
>

> On Thu, Jul 30, 2009 at 5:06 AM, s|s <supreet.se...@gmail.com> wrote:
> > Hi,
> > Just ran few queries on my index which were just exclude type. It ends
> > up with an error <expression> NOT <expression>. It seems one needs
> > FLAG_PURE_NOT

> >http://xapian.org/docs/apidoc/html/classXapian_1_1QueryParser.html#fd...

[Richard Boulton]

On Aug 18, 8:58 pm, notanumber <dnsa...@gmail.com> wrote:
> Just an update on this issue.  FLAG_PURE_NOT should only be set when
> the query string consists of only a NOT expression

FLAG_PURE_NOT will (or at least, should) have no effect when the query
string doesn't consist of a pure NOT expression - the only effect of
the flag is to allow the queryparser to interpret pure NOT queries
such as "NOT query" as "<all documents> NOT query" (without the flag,
this is a syntax error). It should be fine to leave it on in other
cases.

> and it can consume
> quite a bit of memory since it has to load all documents to do so.

It's not quite that bad - it shouldn't actually consume any more
memory than a standard term in a search, since Xapian doesn't work by
loading lots of stuff into memory at once. Also, in many cases (in
particular, if the document IDs in the database are "compact", which
will be the case if you've only ever added documents and not deleted
them), Xapian doesn't need to do any more IO because it's easy to work
out which docids exist. It is likely to take quite a bit more CPU
than other searches, though.

> I've updated the code to only do this when needed.  Let me know if
> it's not behaving as expected.

If your experience with FLAG_PURE_NOT isn't in line with what I've
said above, please let me know, since it probably means there's a bug
in our implementation somewhere.

--
Richard

[David Sauve]

Thanks for the explanation, Richard.  I'll just leave FLAG_PURE_NOT enabled all the time in that case.