zoie Google Group

Out of memory issue

2013-03-26T19:03:15Z

Helllo,
I came across the out of memory issue in my web application. We have
created the heap dump on OOM. After analyzing the heap dump, I figured out
that there are 57 objects of SearchIndexManager.java and 163 object of
BoboIndexReader.java. Which are root cause of the OOM.
While debugging the source code, I figured out that SearchIndexManager

Bobo browse with zoie "cannot overwrite IOexceptipn

2013-01-27T12:09:04Z

Hi,
I am using the bobo browse API. It supports zoie for real time indexing.
Zoie internally using the lucene. I am trying to create index but came
across following error. Any help will be greatly appreciated.
Zoie verison 3.0.0.
Lucene 2.9.2
OS : windows 7 64bit
at org.apache.lucene.store.FSDire ctory.initOutput(FSDirectory.j ava:362)

zoie exception cannot overwrite index file

2013-01-27T12:02:55Z

Hi Everyone,
I am using the bobo-browse with zoie for real time indexing. zoie version
3.0.0, I am getting error message "can not overwrite" I have created jira
ticket. any help will be greatly appreciated.
[link]
Best Regards,
Brij

jira location changed

2012-12-19T23:56:42Z

Hi folks:

We have moved jira location to:
[link]

Thanks

-John

IllegalAccessError with faceted queries

2012-06-18T12:46:16Z

Hi,

Hourglass with zoie 3.1.0 ran smoothly until a while ago and suddenly
the IllegalAccessError is cropping up now, with any faceted query. The
non-faceted queries still run as expected.

The first line of the traceback is "SEVERE:
java.lang.IllegalAccessError: class
org.apache.lucene.index.MultiZ oieTermDocs cannot access its superclass

Re: Problems I found with ZOIE

2012-05-29T05:44:54Z

Hi Zhuobin:

2) is fixed, see the bug for the link to the commit.

I read over the code for 1), and I think what you are saying is indeed
valid.

I opened up a ticket: [link] , and
we can track it from there.

Thank you for your contribution!

How zoie example generate SearchService.js and engine.js files?

2012-05-08T10:55:14Z

I import zoie example to eclipse and deploy to tomcat,No SearchService.js
and engine.js.But i find the both files on the zoie example.I search my
local disk,there are no those files?
How the zoie example generate those files?
thanks

hourglass - archiving expired indexes

2012-04-19T14:04:48Z

Hello,

From the hourglass wiki [1], I see that it is possible to implement
hooks to handle expired index. Has anybody tried this out to archive
the expired indices? Any ideas of how to go about this?

[1]: [link]

Thanks,

init jetty remote jmx with jconsole

2012-04-11T06:00:51Z

I install zoin example on remote linux,I want to init index process on
local win7,I uncomment the block in jxm.spring:
<bean id="httpAdaptor" class="mx4j.tools.adaptor.http .HttpAdaptor"
init-method="start" destroy-method="stop">
*<property name="port" value="7777" />*
* <property name="host" value="192.168.172.29"/>*

while disk index and RAM_A index is mergying, zoie cannot add document to RAM_B?

2012-03-15T04:05:04Z

In class RealtimeIndexDataLoader.java,
we can see method processBatch() is synchronized
and also in the method consume(Collection<DataEvent<D >> events)
there is a synchronized (this) block with comment "this blocks the
batch disk loader thread while indexing to RAM".

Because they are synchronized, we can't execute them at the same time

ZOIE网站上示例在最新版本（zoie-core-3.1.1-SNAPSHOT.jar）下编译会出现一些错误

2012-02-11T11:51:03Z

ZOIE网站上示例在最新版本（zoie-core-3.1.1 -SNAPSHOT.jar）下编译会出现一些错误，烦请哪位修改一下，谢谢！
示例：[link]

3.1.0 released

2012-01-16T23:08:34Z

We have released version 3.1.0, release notes:
[link]
notable: upgraded to lucene 3.5.0
-John

严重: org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed

2012-01-10T02:16:09Z

2012-1-10 10:02:36 org.apache.solr.common.SolrExc eption log
严重: org.apache.lucene.store.Alread yClosedException: this IndexReader
is closed
at
org.apache.lucene.index.IndexR eader.ensureOpen(IndexReader.j ava:256)
at
org.apache.lucene.index.IndexR eader.termDocs(IndexReader.jav a:1000)

ZoieException

2012-01-04T07:08:41Z

I use multiple core of solr ,and when I reload the core , the
exception is throws:

proj.zoie.api.ZoieException: consumer thread has stopped
at
proj.zoie.impl.indexing.AsyncD ataConsumer.consume(AsyncDataC onsumer.java:
245)
at
proj.zoie.solr.ZoieUpdateHandl er.addDoc(ZoieUpdateHandler.ja va:180)

handleUpdatedDiskVersion doesn't work

2011-12-28T06:34:52Z

I has add the addIndexingEventListener in ZoieSystem ,

ZoieSystem _zoie = (ZoieSystem) zoie;
_zoie.addIndexingEventListener (new IndexingEventListener() {
@Override
public void handleIndexingEvent(IndexingEv ent evt) {
log.info(" ###########刷数据到硬盘事件 ###########");
}

proj.zoie.impl.indexing.luceneNRT.LuceneNRTDataConsumer

2011-12-26T08:04:15Z

proj.zoie.impl.indexing.lucene NRT.LuceneNRTDataConsumer<D>

/**
* document ID field name
*/
public static final String DOCUMENT_ID_FIELD = "id";

What 's mean the field?

try {
_writer.deleteDocuments(new Term(DOCUMENT_ID_FIELD, String
.valueOf(indexable.getUID()))) ;
} catch (IOException e) {

Zoie Merge policy

2011-12-21T20:50:52Z

Hi,
I am using the zoie solr plugin and have configured the numlargesegments to
5 and maxsmallsegments to 10, the merge factor is 10 as well. However I
have never seen the number of index files to be less than 15. What I do see
as I keep adding documents is that the number of files keeps growing.
Could you please elaborate on when the optimizer runs and if you are aware

ZoieSegmentReader.getUID maybe throws exceptioin

2011-12-21T09:11:43Z

2011-12-21 16:48:21 org.apache.solr.common.SolrExc eption log
严重: java.lang.ArrayIndexOutOfBound sException: 952
at
proj.zoie.api.ZoieSegmentReade r.getUID(ZoieSegmentReader.jav a:251)
at proj.zoie.solr.ZoieUpdateHandl er
$2.collect(ZoieUpdateHandler.j ava:287)
at org.apache.lucene.search.TermS corer.score(TermScorer.java:

zoie 's RAMDirectory size

2011-12-21T06:06:11Z

public IndexWriter openIndexWriter(Analyzer analyzer, Similarity
similarity) throws IOException {
if (_indexWriter != null)
return _indexWriter;

ZoieMergePolicy mergePolicy = new ZoieMergePolicy();
mergePolicy.setMergePolicyPara ms(_mergePolicyParams);
mergePolicy.setUseCompoundFile (false);

Is this possible to use Zoie in google app engine project

2011-12-11T04:25:48Z

Hi all,
I am new to zoie , Is this possible to use Zoie in google app engine
project?
I am looking for real time search indexing and ranking tool for app
engine java.
Thanks
Deepak

Updating a document (with a new facet value)

2011-11-29T19:15:38Z

An example will probably be easier:
Say for example I have a user and I want to store what schools he went
to (for facetted search)
I would use a multivalue facet field in my user document, for example
it would be called user_schools - It will store the school names.
Now i can create that document fine, but I'm not sure on what is the

Zoie UId

2011-11-14T19:35:47Z

It seems like Zoie api's(ZoieIndexable) requires a unique document UID
as a long, could it be possible to use a string UID (we currently use
a java UUID)

If not that's ok, I guess we can figure something out.

Thanks

Hi ,all. This is about the result of my zoie test

2011-10-18T16:08:28Z

*NRT *: *TPS is *800-1000K/S
*ZOIE: TPS is half of NRT'*
I used a profile tool to get the cost of methods in Zoie's Index package.
I found that:
proj.zoie.impl.indexing.intern al.BaseSearchIndex.updateIndex () costs a
lot;
in updateIndex, addDocument()'s cost is 50%,
IndexWriter.commit() / openIndex()/ closeIndex() totally cost is 50%

Zoie powering Bit.ly

2011-10-14T19:29:29Z

Hi guys:

I just learned from Dennis Kubes, search architect at Bit.ly that
the newly announced Bit.ly social search engine is built on Zoie!

Congrats and thanks to everyone contributed to Zoie!

-John

ZoieIndexable responsability

2011-10-13T02:34:39Z

Hi,
I can't figure out why a ZoieIndexable can respond to messages:
getUID
getIndexingReq[]
To me, that means that many documents can (and will, based on all
DataConsumer implementation I've seen so far) map to a single UID.
Which means, following events with same UID will delete all of these
documents from the index.

Zoie and solr 3.3.0 and HourGlass?

2011-09-30T01:28:57Z

Hi folks, I've been running an older version of solr 1.41, with a
trunk release solr/hourglass.

My question is with zoie now supporting lucene 3.3.0, does that 'mean'
I should be able to upgrade to solr 3.3.0, with hourglass working?

I'm guessing..yes? but has anyone confirmed the 3 pieces working

Does mergeFactor not work?

2011-09-29T02:47:21Z

There are two threads in my program.
one creates dataEvent and consumes it by calling
zoieSystem.consume(envet), the data is from a file, the logic is like
this :

while( (text= filereader.readLine() )!=null ){
.... create data event with text....
zoieSystem.consume(dataEvent);
zoieSystem.syncWithVersion(100 00, version);

zoie-solr unique key - string or long?

2011-09-23T03:08:35Z

Hi everyone.
I just installed Solr3.4 with zoie3.0.
The problem is that, Solr requires unique key to be of string type,
while zoie's unique key has to be long integer.
Anyone knows how to solve the conflict?

Zoie 3.1.0 roadmap

2011-09-19T15:30:49Z

Hi folks:

We are in the process of putting together a roadmap for 3.1.0
release: [link]

Please take sometime and go through the unresolved list and pick
ones you feel should be in this release, or create issues that you
feel should be in it.

homepage upgraded

2011-09-17T04:59:37Z

Our homepage has gone through a make-over:

[link]

-John

lucene 3.3.0 upgrade work completed

2011-09-08T17:21:27Z

Hi:

I have completed to work to make zoie compatible with lucene 3.3.0
and all unit tests now pass.

It is currently on the branch lucene-upgrade, and will merge down
to master later on today.

-John

Committer team formed

2011-09-03T03:04:03Z

Hi guys:

The Zoie committer team has been established:

Brian Hammond (internap.com)
Dennis Kubes (bit.ly)
Jake Mannix (twitter)
Xiaoyang Gu (linkedIn)
Yasuhiro Matsuda (linkedIn)
Baoqiu Cui (linkedIn)
Sheng Guo (linkedIn)
John Wang (linkedIn)
Lei Wang (linkedIn)
Hao Yan (linkedIn)

IndexReaderFactory problem

2011-08-31T23:11:01Z

Hello,

I have a noobie question here. I am trying to following this
instruction to setup zoie:

[link]

I copied all four required jar files into: C:\Solr\apache-
solr-3.3.0\example\lib

When I start the solr server, I got the following error message:

Naive question: Is Zoie really supports real-time search?

2011-08-26T06:31:31Z

Hello,

I've started learning Zoie recently, and faced with "trivial" problem
- my indexed documents are not visible for immediate search without
explicit calls of flushEvents() / flushEventsToMemoryIndex() /
syncWithVersion.

I found the similiar topic in this group, but things recomended there

Noob Questions

2011-08-13T11:27:27Z

Hey everyone,
I have only been looking into Zoie for only a little while now so forgive my
noobness.
I have been looking at the limited documentation I found:
[link]
[link]
But it stil does not give an excellent idea of the structure of Zoie so as

Zoie + Solr plugin

2011-08-03T00:53:14Z

I have posted the same question couple of times, once in actual zoie
site and also have emailed the owners.
I am not sure whether that would reach you guys (spam filters or
something).

Apologies, this might actually be considered as spam :), posting it
everywhere.

Have faced problems integrating zoie with solr using zoie-solr plugin.

A newbie question

2011-07-20T07:05:55Z

Hi all ,

I have following doubts

1)Should zoie indexwriter and zoie indexreader needs to be on the same
JVM (Same process) so as to achieve the speed zoie promises .

2)Will zoie uploads the entire index directory to main memory or is it
some index file to the index directory files ? My main concern is that

Deleting Documents Failed

2011-07-14T02:49:44Z

When I flush a delete doc data Event, I found it didn't work.
When I dived into the codes, I found the reader object below is null. So I
cannot delete docs. How can I make it work? Thank you.
=================BaseSearchInd ex.java======================= ==============
private void deleteDocs(LongSet delDocs) throws IOException

How to manage a single ZoieSystem instance throughout the application

2011-07-13T13:19:38Z

How can i manage a single ZoieSystem instance throughout my
application? can anyone please explain? i don't understand from the
search example.

Sorting in Zoie 2.6.0

2011-07-12T22:20:38Z

Hi All,
I have been trying to get a sort to work in zoie 2.6.0 but for the life
of me can't get it to work.
The fields I am trying to sort by are datestrings in yyymmddhhmiss
format. I am sorting them as strings. They are all Stored and not
analyzed. Example:
doc.add(new Field(Fields.INITIAL, initialSeen, Field.Store.YES,

JDBC StreamDataProvider help

2011-07-08T05:32:33Z

Hi,

JDBCStreamDataProvider dataprovider=new
JDBCStreamDataProvider(confact ory,mybuilder,versioncomp);

dataprovider.setDataConsumer(i ndexingSystem);
dataprovider.start();
indexingSystem.flushEvents(100 00);

Should I do anything more than this to get my indexDirectory created
using JDBC Stream provider?

How can i do indexing and searching in two different classes ?

2011-07-07T05:54:33Z

I am writing my search code like this,
List<ZoieIndexReader<MyDoNothi ngFilterIndexReader>> readerList =
indexingSystem.getIndexReaders ();

but how can i access the current instance of indexingSystem , as i am
writing search code in another class.

OR is there any way to implement search without affecting the indexing

[zoie]what does "ZoieVersion" stand for in zoie?

2011-07-06T07:50:54Z

Thanks John and Gu's great contribution to zoie. I am build a search server
based on zoie, but I am confused about the concept of "Version". Why do we
need it? My index has a lot of update operations, so do I have to remember
the value of last Data Event?

Trouble opening an index directory via ZoieIndexReader

2011-07-05T21:33:20Z

I'm having trouble opening an existing index directory on my disk
using ZoieIndexReader. I'm creating it as :

FSDirectory dir1 = FSDirectory.open(new File("index_dir"));
indReader1 = IndexReader.open(dir1,true);
zIndReader1 = ZoieIndexReader.open(indReader 1);

After this however, it seems that delDocIds and docIDMapper are NULL.

How to use Zoie with multiple different indexes

2011-07-05T15:19:53Z

Hi All,

I was wondering on how to use Zoie with Multiple indexes. Do I need to
create that many multiple Zoie instances as the number of different
indexes I want. or is there a simpler way to do it.

--Thanks and Regards
Vaijanath

Zoie Examples

2011-07-05T09:11:27Z

Hi All,
I am new to Zoie so was wondering if there is any example any tutorial which
speaks on how to use Zoie in production environment.
--Thanks and Regards
Vaijanath N. Rao

How Can i write a PreparedStatementBuilder ?

2011-07-05T10:47:23Z

I am trying to set up JDBCStreamDataProvider. how can i write a
PreparedStatementBuilder ? How can i iterate without using
Result.next() ? . I wrote a PreparedStatementBuilder, which is given
below but i am not getting indexDirectory created.

dataprovider.setDataConsumer(i ndexingSystem);
dataprovider.start();

When is actually the index directory getting created?

2011-07-04T12:04:27Z

I am trying the sample codes but no index directory is getting created
in the location i specified, File idxDir = new File("/home/developer/
NetBeansProjects/Zoie/myIdxDir ");

when is actually the indexDirectory getting created? is it created
when we call indexingSystem.start(); ? or
indexingSystem.consume(eventLi st); ? what are all the general mistakes

ZoieSystem constructor not found

2011-07-01T07:59:33Z

I am using Zoie2.6.0 , when i am trying to create a ZoieSystem as
mentioned in the code samples ,
ZoieSystem indexingSystem = new
ZoieSystem(idxDir,myInterprete r,decorator,analyzer,similarit y,
1000,300000,true);

it says cannot find symbol constructor. Can anyone help me with
this ??? I am a newbie

About Index Maintain

2011-06-29T10:30:38Z

How do you maintain your index files in ZOIE ? Do you need to completely
rebuild your index files monthly?
In my search system based on lucene 2.9.3, I have to completely rebuild the
index files every two weeks so as to clean and make index files smaller.