Does mergeFactor not work?

[Bob]

There are two threads in my program.
one creates dataEvent and consumes it by calling
zoieSystem.consume(envet), the data is from a file, the logic is like
this :

while( (text= filereader.readLine() )!=null ){
.... create data event with text....
zoieSystem.consume(dataEvent);
zoieSystem.syncWithVersion(10000, version);
version++;
}

the other thread searches the index by calling
zoieSystem.getIndexReaders();

The problem is that there are many small segments in the index
directory ,it seems like that the segments are never merged.
The param mergeFactor is 6 in ZoieMergePolicy .

[John Wang]

Hi Bob:

    For realtime indexing scenarios, we want to avoid large segment merges. So instead of optimizing for 1 large segment, zoie optimizes for N balanced segments and thus spread the merge cost over time.

     The configuration parameters are number of Large segments and number of Smal segments. You should only be tuning those parameters. I would suggest you leave mergeFactor alone. For us, our production system has been running rather smoothly without changing the default parameters.

 -John

--
You received this message because you are subscribed to the Google Groups "zoie" group.
To post to this group, send email to zo...@googlegroups.com.
To unsubscribe from this group, send email to zoie+uns...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/zoie?hl=en.

[Bob]

I think the reason is that the writer.addIndexesNoOptimize()
function is changed in lucene3.3;
In Lucene 3.0 , addIndexesNoOptimize() maybe cause merge. In
Lucene3.3, it does not.
Maybe this is a bug .

[Bob]

我把程序改成使用 zoie3.0 based on Lucene.3.0 ， 现在正常了，只有10几个segment了。

建议新版本的zoie 使用 indexWriter.addIndexes(IndexReader... readers) 这个方法。

[John Wang]

Thanks Bob!

The latest Zoie 3.0.0 release is based on Lucene 3.0.

Are you seeing this with latest on master?

I have created: http://linkedin.jira.com/browse/ZOIE-88 to track this. Thanks for pointing this out!

-John

[John Wang]

Hi Bob:

    I am the fix with this commit: https://github.com/javasoze/zoie/commit/8d591d08682cbc7313046889a91a0578a4afb46e

    And also published a snapshot jar (3.1.0-SNAPSHOT) to maven. Please give it a try and let me know if it is better.

    Thanks again for pointing this out!

-John