Whoosh index problem

[Rama Vadakattu]

iam having the following problem with whoosh full text search engine.

1.After syncdb i am creating the intial index from the database
objects.

2.it is working fine.I can able to search the data and see the
results.

3.after that in one of my view i have added another document (via
signals) to the index (during a request --response)

4.that' it from then onwards i could not able to search any data , for
which i have successfully found results before adding new document
(before step 3)

ix = storage.open_index()
writer = ix.writer()
writer.add_document(.............)

I have tried hard to resolve but i could not.

Any ideas on how to resolve this problem?

[Ruben Bakker]

Did you commit your changes? 

In addition to:

ix = storage.open_index()

writer = ix.writer()
writer.add_document(.............)

I use these two lines to commit the changes:

writer.commit()

ix.close()

--
Ruben
http://mailplaneapp.com/blog
http://www.twitter.com/Mailplane

[Rama Vadakattu]

yes iam doing that.................

sample code for your reference.
http://dpaste.com/hold/86502/

rama

On Aug 28, 2:29 pm, Ruben Bakker <ruben.bakk...@gmail.com> wrote:
> Did you commit your changes?
>
> In addition to:
> ix = storage.open_index()
> writer = ix.writer()
> writer.add_document(.............)
>
> I use these two lines to commit the changes:
>
> writer.commit()
>
> ix.close()
>
> On Fri, Aug 28, 2009 at 11:24 AM, Rama Vadakattu

> <rama.vadaka...@gmail.com>wrote:

[Rama Vadakattu]

Some more finding.............

After building initial index ,
the moment i write the another document to the index ,
index is getting cleared thereafter i am not getting
the results for the any search.

writer2.add_document(name=u"asdf",content=u"asdf
asdf",instituteid=u"2234234",id=u"32asdf345545")
write2.commit()

Any one encountered this problem?

On Aug 28, 2:33 pm, Rama Vadakattu <rama.vadaka...@gmail.com> wrote:
> yes iam doing that.................
>

> sample code for your reference.http://dpaste.com/hold/86502/

[Matt Chaput]

Rama Vadakattu wrote:
> 4.that' it from then onwards i could not able to search any data , for
> which i have successfully found results before adding new document
> (before step 3)

What do you mean by "not able to search any data"? Is there an error, or
are the results empty? In the listing you put on dpaste, the actual
adding of documents is hidden inside an "adddoc" function. Anything
interesting going on in there? Can you reduce the problem to a test case?

Thanks,

Matt

[Rama Vadakattu]

matt,

1.the results are empty.

2.before adding the document thre results were comming properly.

3.After adding the results were empty.

3.I will prepare a test case and will post here immediately.

--rama

[Rama Vadakattu]

matt,

1. i could not able to reproduce the problem when i was trying to
replicate it on the command line.

2. adddoc code http://dpaste.com/hold/86874/

(just see the last two lines it is just adding/updating a document
to the index)

3.Don't know what can be the problem in my code. (Any ideas of what
can go wrong?)

--rama

[Matt Chaput]

> 1. i could not able to reproduce the problem when i was trying to
> replicate it on the command line.

I'm guessing it has something to do with merging. So it would only
show up when you create a writer and add documents more than once, e.g.

w1 = index.writer()
w1.add_document(...)
w1.commit()
w2 = index.writer()
w2.add_document(...)
w2.commit()

I'll try to reproduce the problem here this weekend. I hope we can get
to the bottom of this quickly.

Thanks!

Matt

[Matt Chaput]

>> 1. i could not able to reproduce the problem when i was trying to
>> replicate it on the command line.
>
> I'm guessing it has something to do with merging. So it would only
> show up when you create a writer and add documents more than once,
> e.g.

I should have said, either merging, or multi-searching (where the
index has multiple segments and the reader composites them). I'll try
to recreate both conditions and see if I can see what you're getting.

Mat

[Rama Vadakattu]

Thanks Matt

> > I'm guessing it has something to do with merging. So it would only
> > show up when you create a writer and add documents more than once,
> > e.g.

Just to let you know:
i have added documents twice on command line.
it is working properly

--rama

[Matt Chaput]

> Thanks Matt
>
>>> I'm guessing it has something to do with merging. So it would only
>>> show up when you create a writer and add documents more than once,
>>> e.g.
>
> Just to let you know:
> i have added documents twice on command line.
> it is working properly

I've got a test case, it seems to be a problem with MultiReader
(yikes!). Will try to figure out a fix right away.

Matt

[Rama Vadakattu]

Thanks Matt. ( i have been working hard from yesterday to find what
can be the problem..................).


--rama

[Shie Erlich]

was this ever?

i'm encountering the exact same problem on 2.7.2

[Shie Erlich]

was this ever => was this ever fixed? 

[Shie Erlich]

actually - i take it back. 

what i'm seeing is that the previous results are still there, yet the new document doesn't appear to get indexed somehow as i can't find any matches in it.

apologies for the confusion!

[Matt Chaput]

> actually - i take it back.
> what i'm seeing is that the previous results are still there, yet the new document doesn't appear to get indexed somehow as i can't find any matches in it.

OK, well, if there’s a bug, please create an issue for it.

The best first step when something isn’t being found is to check what was actually indexed (e.g. use searcher().lexicon(fieldname)) vs. what you’re searching for.

Cheers,

Matt

[Shie Erlich]

Matt - thanks for your kind reply. 

this morning, i was able to finally fix this and it was NOT a whoosh issue after all, but rather a bug in my code :(

apologies for the false alarm!