nothig happened with Zoie-solr plugin

33 views
Skip to first unread message

think

unread,
Apr 24, 2010, 1:32:37 PM4/24/10
to zoie
I also use Zoie-solr plugin with tomcat, and config it by
http://snaprojects.jira.com/wiki/display/ZOIE/Zoie+Solr+Plugin. but
nothing happened. here is my solrconfig.xml.

<config>
...

<indexReaderFactory
class="proj.zoie.solr.ZoieSolrIndexReaderFactory" />

<updateHandler class="proj.zoie.solr.ZoieUpdateHandler">
</updateHandler>

<zoie.batchSize>1000</zoie.batchSize> <!-- default: 1000 -->
<zoie.batchDelay>60000</zoie.batchDelay> <!-- default: 300000
e.g. 5 min -->
<zoie.realtime>true</zoie.realtime> <!-- default: true -->

<uniqueKey>id</uniqueKey> <!-- mysql table
primary key -->

...

</config>

is there anything wrong? thanks for you help.

--
You received this message because you are subscribed to the Google Groups "zoie" group.
To post to this group, send email to zo...@googlegroups.com.
To unsubscribe from this group, send email to zoie+uns...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/zoie?hl=en.

Jake Mannix

unread,
Apr 24, 2010, 3:35:41 PM4/24/10
to zo...@googlegroups.com

When the zoie-solr plugin is installed with your Solr instance, you still need to update as usual to Solr.   It doesn't have any special hooks to "pull from a DB" or anything, you still index and search solr as usual.

  -jake

think

unread,
Apr 24, 2010, 9:00:41 PM4/24/10
to zoie
thanks for your reply. do you mean I should write a service or other
to trigger update event periodically? I want it can update automatic,
is there other solution?
> zoie+uns...@googlegroups.com <zoie%2Bunsu...@googlegroups.com>.

think

unread,
Apr 24, 2010, 9:13:43 PM4/24/10
to zoie
there is other question. in my solrconfig,xml.

<uniqueKey>id</uniqueKey> <!-- mysql table primary
key -->

if there are multi tables(mysql) need to be indexed, every table have
a primary key. how to set the uniquekey at solrconfig.xml?

<uniqueKey>id1</uniqueKey> <!-- mysql table1
primary key id1 -->
<uniqueKey>id2</uniqueKey> <!-- mysql table2
primary key id2 -->
<uniqueKey>id3</uniqueKey> <!-- mysql table3
primary key id3 -->

is it right?
> zoie+uns...@googlegroups.com <zoie%2Bunsu...@googlegroups.com>.

Jake Mannix

unread,
Apr 24, 2010, 11:40:57 PM4/24/10
to zo...@googlegroups.com
On Sat, Apr 24, 2010 at 6:00 PM, think <gospe...@gmail.com> wrote:
thanks for your reply. do you mean I should write a service or other
to trigger update event periodically? I want it can update automatic,
is there other solution?

What do mean that you want it to update automatically?  Do you mean
that when you change a database row, that it automatically gets indexed?
With Solr, there is the concept of a DataImportHandler, which can be
set up to pull via JDBC from a database.  In Zoie, there is the concept
of a JDBCStreamDataProvider, which can do similarly.

Unfortunately, the zoie-solr plugin does not expose the latter method
via configuration, so the only way to go right now is to have a traditional
Solr DataImportHandler (ie the <dataSource> tag), then zoie
will take it from there and make the data available as soon as it hits
the Solr instance.

Writing your own service to push changes immediately from the DB
over to your Solr instance is another option, as that would minimize
the latency / turnaround between these changes hitting the DB and
being visible in the index.

  -jake

That doesn't come built-in with either Zoie or Solr, you need to hook up
a Solr DataImportHandler to get data events from the database into 
your Solr instance.  

Jake Mannix

unread,
Apr 24, 2010, 11:43:01 PM4/24/10
to zo...@googlegroups.com
On Sat, Apr 24, 2010 at 6:13 PM, think <gospe...@gmail.com> wrote:
there is other question. in my solrconfig,xml.

<uniqueKey>id</uniqueKey>                     <!-- mysql table primary
key -->

if there are multi tables(mysql) need to be indexed, every table have
a primary key. how to set the uniquekey at solrconfig.xml?

Each Zoie instance can only have *one* unique key.  If you need to
index multiple tables with zoie-solr, you need to set up Solr with
"multi-core" support, and have each table in a separate solr-core.

  -jake

John Wang

unread,
Apr 25, 2010, 2:08:58 AM4/25/10
to zo...@googlegroups.com
zoie-solr plugin does not work with Solr multi-core, see: https://issues.apache.org/jira/browse/SOLR-1807

-John

John Wang

unread,
Apr 26, 2010, 2:41:49 AM4/26/10
to zo...@googlegroups.com
Ok, so I gave a try to the autocommit implementation and failed miserably!
The problem is that the solr caches searchers, not readers! This is really bad, so creation of the searcher is really slow, and cannot handle the update load zoie is able to handle.

I will look more into how to hack solr. But not very optimistic at this point.

-John

bharath venk

unread,
Apr 26, 2010, 3:37:58 AM4/26/10
to zoie
Hi John,

Did you take care of auto warm count of solr caches (query
result cache , filter cache ,document cache ) .. if not , I guess
auto warm count of all solr caches should be set to zero for near
real time search other wise it would try to warm the cache every time
the commit happens ..

Thanks,

On Apr 26, 11:41 am, John Wang <john.w...@gmail.com> wrote:
> Ok, so I gave a try to the autocommit implementation and failed miserably!
> The problem is that the solr caches searchers, not readers! This is really
> bad, so creation of the searcher is really slow, and cannot handle the
> update load zoie is able to handle.
>
> I will look more into how to hack solr. But not very optimistic at this
> point.
>
> -John
>
>
>
> On Sat, Apr 24, 2010 at 11:08 PM, John Wang <john.w...@gmail.com> wrote:
> > zoie-solr plugin does not work with Solr multi-core, see:
> >https://issues.apache.org/jira/browse/SOLR-1807
>
> > <https://issues.apache.org/jira/browse/SOLR-1807>-John
>
> > On Sat, Apr 24, 2010 at 8:43 PM, Jake Mannix <jake.man...@gmail.com>wrote:
>
> >> On Sat, Apr 24, 2010 at 6:13 PM, think <gospelan...@gmail.com> wrote:
>
> >>> there is other question. in my solrconfig,xml.
>
> >>> <uniqueKey>id</uniqueKey>                     <!-- mysql table primary
> >>> key -->
>
> >>> if there are multi tables(mysql) need to be indexed, every table have
> >>> a primary key. how to set the uniquekey at solrconfig.xml?
>
> >> Each Zoie instance can only have *one* unique key.  If you need to
> >> index multiple tables with zoie-solr, you need to set up Solr with
> >> "multi-core" support, and have each table in a separate solr-core.
>
> >>   -jake
>
> >>> <uniqueKey>id1</uniqueKey>                     <!-- mysql table1
> >>> primary key id1 -->
> >>> <uniqueKey>id2</uniqueKey>                     <!-- mysql table2
> >>> primary key id2 -->
> >>> <uniqueKey>id3</uniqueKey>                     <!-- mysql table3
> >>> primary key id3 -->
>
> >>> is it right?
>
> >>> Jake Mannix wrote:
> >>> > When the zoie-solr plugin is installed with your Solr instance, you
> >>> still
> >>> > need to update as usual to Solr.   It doesn't have any special hooks to
> >>> > "pull from a DB" or anything, you still index and search solr as usual.
>
> >>> >   -jake
>
> >>> zoie%2Bunsu...@googlegroups.com<zoie%252Buns...@googlegroups.com>
> For more options, visit this group athttp://groups.google.com/group/zoie?hl=en.

John Wang

unread,
Dec 15, 2011, 11:31:37 PM12/15/11
to genuine, zoie
+Zoie mailing list

What is Solr-RA?

-John

2011/12/15 genuine <yzyge...@gmail.com>
The solr-ra company has hack the solr,It seems more effected to
visible as soon as possible..

when some document update ,it will open a new reader ,change the
searcher 's reader,so the currentSearcher can get the lastest reader .


On 2010年4月26日, 下午2时41分, John Wang <john.w...@gmail.com> wrote:
> Ok, so I gave a try to the autocommit implementation and failed miserably!
> The problem is that the solr caches searchers, not readers! This is really
> bad, so creation of the searcher is really slow, and cannot handle the
> update load zoie is able to handle.
>
> I will look more into how to hack solr. But not very optimistic at this
> point.
>
> -John
>
>
>
>
>
>
>
>
>
> On Sat, Apr 24, 2010 at 11:08 PM, John Wang <john.w...@gmail.com> wrote:
> > zoie-solr plugin does not work with Solr multi-core, see:
> >https://issues.apache.org/jira/browse/SOLR-1807
>
> > <https://issues.apache.org/jira/browse/SOLR-1807>-John
>
> > On Sat, Apr 24, 2010 at 8:43 PM, Jake Mannix <jake.man...@gmail.com>wrote:

>
> >> On Sat, Apr 24, 2010 at 6:13 PM, think <gospelan...@gmail.com> wrote:
>
> >>> there is other question. in my solrconfig,xml.
>
> >>> <uniqueKey>id</uniqueKey>                     <!-- mysql table primary
> >>> key -->
>
> >>> if there are multi tables(mysql) need to be indexed, every table have
> >>> a primary key. how to set the uniquekey at solrconfig.xml?
>
> >> Each Zoie instance can only have *one* unique key.  If you need to
> >> index multiple tables with zoie-solr, you need to set up Solr with
> >> "multi-core" support, and have each table in a separate solr-core.
>
> >>   -jake
>
> >>> <uniqueKey>id1</uniqueKey>                     <!-- mysql table1
> >>> primary key id1 -->
> >>> <uniqueKey>id2</uniqueKey>                     <!-- mysql table2
> >>> primary key id2 -->
> >>> <uniqueKey>id3</uniqueKey>                     <!-- mysql table3
> >>> primary key id3 -->
>
> >>> is it right?
>
> >>> Jake Mannix wrote:
> >>> > When the zoie-solr plugin is installed with your Solr instance, you
> >>> still
> >>> > need to update as usual to Solr.   It doesn't have any special hooks to
> >>> > "pull from a DB" or anything, you still index and search solr as usual.
>
> >>> >   -jake
>

genuine

unread,
Dec 16, 2011, 12:56:51 AM12/16/11
to zoie
http://solr-ra.tgels.com/

On Dec 16, 12:31 pm, John Wang <john.w...@gmail.com> wrote:
> +Zoie mailing list
>
> What is Solr-RA?
>
> -John
>

> 2011/12/15 genuine <yzygenu...@gmail.com>

genuine

unread,
Dec 16, 2011, 5:04:26 AM12/16/11
to zoie
what 's the zoie mailing list ?

On Dec 16, 12:31 pm, John Wang <john.w...@gmail.com> wrote:

> +Zoie mailing list
>
> What is Solr-RA?
>
> -John
>

> 2011/12/15 genuine <yzygenu...@gmail.com>

Reply all
Reply to author
Forward
0 new messages