Indexing

[alek...@gmail.com]

Hi,

Currently I cannot perform a search in my projects since the index building is in progress and the indexing seems to never stop. 

I do not have much admin experience, so could you give me a hint, what I should check?  The logs only show that the indexing is in progress and no further details. Could it be, the indexing is just hanging due to a problem with a single file or the index cannot be written? Where do I find the lucene index file?

Best regards

Aleksandra

[Richard Eckart de Castilho]

On 23. Jul 2021, at 11:25, alek...@gmail.com wrote:
>
> Currently I cannot perform a search in my projects since the index building is in progress and the indexing seems to never stop.
>
> I do not have much admin experience, so could you give me a hint, what I should check? The logs only show that the indexing is in progress and no further details. Could it be, the indexing is just hanging due to a problem with a single file or the index cannot be written? Where do I find the lucene index file?

What version are you using?

-- Richard

[alek...@gmail.com]

I use 0.19.7.

--
You received this message because you are subscribed to the Google Groups "inception-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to inception-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/inception-users/221E444E-AF87-4B42-A761-3164013BE7D4%40gmail.com.

[Richard Eckart de Castilho]

On 23. Jul 2021, at 11:33, alek...@gmail.com wrote:
>
> I use 0.19.7.

Are you using a knowledge base - a remote one in particular?

-- Richard

[alek...@gmail.com]

Hi Richard,

there were 3 recommenders in various projects, 1 simple StringMatcher and  2 remote classifiers. However they were disabled or the service was not working anyway. I deleted all of them and restarted inception.  

I still have the same problem, especially in one bigger project, which contains almost 10k documents. Could the number of docs be the problem? However It worked before.

Aleks

Aleksandra

--
You received this message because you are subscribed to the Google Groups "inception-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to inception-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/inception-users/0EE2F7AB-A66A-442B-9330-EEF837B79DE4%40gmail.com.

[alek...@gmail.com]

Hi Richard,

sorry that I am asking one more time, but the search function is currently very important for me.

Do you think that the number of documents in our project could cause the problem with indexing? The problematic project currently has almost 11k documents. The whole inception database contains about 27k documents. After the search failed due to indexing  again I started rebuilding the index, then waited for 40 minutes  and tried unsuccessfully to search again (the indexing is still in progress). How long should it take?  

Or could the indexing problem be caused by adding more features to existing layers? That does not make sense to me, but it is the only recent modification I can think of besides adding a very few more documents. No recommenders or knowledge bases are currently used.

Maybe you do have any more idea what could cause the problem? 

Best regards,

Aleksandra

[Richard Eckart de Castilho]

Hi Aleksandra

sorry, forgot to get back to you after the 20.0 release...

> On 2. Aug 2021, at 12:32, alek...@gmail.com wrote:
>
> Do you think that the number of documents in our project could cause the problem with indexing? The problematic project currently has almost 11k documents. The whole inception database contains about 27k documents. After the search failed due to indexing again I started rebuilding the index, then waited for 40 minutes and tried unsuccessfully to search again (the indexing is still in progress). How long should it take?
>
> Or could the indexing problem be caused by adding more features to existing layers? That does not make sense to me, but it is the only recent modification I can think of besides adding a very few more documents. No recommenders or knowledge bases are currently used.
>
> Maybe you do have any more idea what could cause the problem?

I cannot really say how long the indexing will take in your case, but I expect it could indeed take a very long time.

That said, in order to get some insight on your case, I added a functionality in INCEpTION 20.0 which gives you
feedback about the indexing process. When you try a search and it is cancelled because indexing is in progress,
the message will now say how many documents have already been indexed and how many are to be indexed in total.
While it doesn't make an estimate how much time indexing will take in total, it will at least give you a progress
indication. I hope that will help you a bit further in this case.

So please try INCEpTION 20.0. Mind that the new version makes changes to the database scheme so that you cannot switch
back to 0.19.7 after you have started 20.0. It is always a good idea to keep disk/DB backups and/or regularly export
projects in order to have backups of them.

Best,

-- Richard

[alek...@gmail.com]

Hi Richard,

ok, I will be able to upgrade to the 0.20.0 release next week and then I will let you know if the problem will still occur.

After some more talks with the annotators I don't think the number of documents is the main issue, since the search seemed to work immediately as we had 10k docs  and the problem started to occur with only a few documents more.
However the functionality of showing the indexing progress, which you added to the latest version might be very helpful to find the problem.

Thanks for your help.

Best regards, Aleksandra

--
You received this message because you are subscribed to the Google Groups "inception-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to inception-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/inception-users/3E8CDD9A-C276-41B8-9428-E82AE959EBBF%40gmail.com.

[Richard Eckart de Castilho]

On 3. Aug 2021, at 11:37, alek...@gmail.com wrote:
>
> After some more talks with the annotators I don't think the number of documents is the main issue, since the search seemed to work immediately as we had 10k docs and the problem started to occur with only a few documents more.
> However the functionality of showing the indexing progress, which you added to the latest version might be very helpful to find the problem.

If the indexing somehow stops, maybe look out in the server logs for an exception.

-- Richard