Integrating MongoDB with Solr

[ken]

Has anyone recently been using Solr to index data in MongoDB?
If so, how are you configuring the Solr data-import handler to accept
the data?

If we can figure out a stable solution for this, then we might be
interested in switching from Postgres to MongoDB. The search doesn't
have to be real-time for us (though near real-time would be nice).

Cheers,
Ken

[Eliot Horowitz]

I don't think there is a "preferred" solution, but I think this is a
great space for someone to right a nice library. Would be pretty easy
to hook in the MongoDB oplog into solr for pseudo real time indexing,
etc...

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>
>
>

[stunti]

so far I did it by creating an indexer in PHP which fetch data from
MongoDB and index them in Solr. Not efficient and not real time. I
would be really happy to have a better solution.

On Jan 19, 11:03 am, Eliot Horowitz <eliothorow...@gmail.com> wrote:
> I don't think there is a "preferred" solution, but I think this is a
> great space for someone to right a nice library.  Would be pretty easy
> to hook in the MongoDB oplog into solr for pseudo real time indexing,
> etc...
>

[Damon P. Cortesi]

I've been waiting for a preferred solution as well. Just because I've
had previous experience with it before, I forked a ruby sphinx library
and have extended that to do what I need for FTI.

http://github.com/dacort/mongosphinx

I did play around with Solr briefly at one point and just configured
my ruby models such that when a change occurred, the necessary XML
data was posted to the Solr server.

[tung vu duc]

In Java world is Compass (from Shay Banon, a architect at GigaSpace) a favorited OSEM Framework (Object to Search Engine Mapper), it's Backend is Lucene. It was seamless integrated with many SQL DBs over Hibernate. I think, it has abstract architecture/design to work with many types of underlying Persistence System, not only RDBMS.

Specially, it has support for Clustering Lucene indexes over Terracotta, EHCache or GigaSpace.

---

From: Eliot Horowitz <elioth...@gmail.com>
To: mongod...@googlegroups.com
Sent: Tue, January 19, 2010 4:03:14 AM
Subject: Re: [mongodb-user] Integrating MongoDB with Solr

I don't think there is a "preferred" solution, but I think this is a
great space for someone to right a nice library.  Would be pretty easy
to hook in the MongoDB oplog into solr for pseudo real time indexing,
etc...

On Mon, Jan 18, 2010 at 5:20 PM, ken <krei...@gmail.com> wrote:
> Has anyone recently been using Solr to index data in MongoDB?
> If so, how are you configuring the Solr data-import handler to accept
> the data?
>
> If we can figure out a stable solution for this, then we might be
> interested in switching from Postgres to MongoDB. The search doesn't
> have to be real-time for us (though near real-time would be nice).
>
> Cheers,
> Ken
>
> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.

> To unsubscribe from this group, send email to mongodb-user+unsub...@googlegroups.com.

> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>
>
>
--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.

To unsubscribe from this group, send email to mongodb-user+unsub...@googlegroups.com.

[joe kim]

Would it be possible to use the mongo replication protocol to update
solr instead of having to poll mongo periodically?

Joe

>> mongodb-user...@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/mongodb-user?hl=en.
>>
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to

> mongodb-user...@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/mongodb-user?hl=en.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to

> mongodb-user...@googlegroups.com.

[Eliot Horowitz]

Yes. I think that would be the best way of doing it actually

[Damon P. Cortesi]

Hi Ken,

I just did this over the weekend. Not sure what language you're using,
but I'm in ruby and ended up using the sunspot-solr library and
defining the appropriate adapters. As far as I know, sunspot-solr just
POSTs documents to the solr HTTP server with the appropriate XML.
Tapping into the replication protocol would be much better, but this
works OK for the moment.

Damon

On Jan 18, 2:20 pm, ken <kreis...@gmail.com> wrote:

[Al Cripps]

Damon,
Could you post additional information on how you did the integration and/or provide additional information  else where or contact me at my email address?  This is something that I have to do in the near future for a Rails application and Sunspot :-)
thanks,
al

[Ericson Smith]

We're using Ruby with MongoMapper and Solr.

The rsolr gem is a bit older, but pretty neat. We tapped into before_create, after_update and after_delete mongomapper hooks to use rsolr to send update and delete calls over http to the solr server.

Solr runs on a biggish server here, and in our dev environment, we're using it to pretty much show any screen that needs some kind of browse/filter/search/pagination, thus bypassing MongoDB entirely for any task not including create, update, delete and view (no list screens call upon mongo!).

So we have pretty much real-time search with this setup (a couple of seconds at most delay for anything).

It took us like an hour or two to write a simple solr class to do updates and deletes, and to hook it into our mongomapper models. We use rsolr to do searches too, which works out great.

- Ericson Smith

W3matter LLC

CTO: http://www.funadvice.com

--

[Damon P. Cortesi]

Hi Al,

This was a quick weekend project I did this on, so it's not fully
integrated and not tied into Rails yet, but I posted about the process
here:
http://www.untitledstartup.com/2010/01/how-to-use-twitter-lists-to-determine-influence/

The important parts are these two code snippets:
http://gist.github.com/287188 - MongoMapper models
http://gist.github.com/287191 - Actual Sunspot solr config for mongo

There's a sunspot-rails project as well that I tried to hack into
being usable for MongoMapper, but I wasn't familiar enough with the
code at the time to make it happen. I may take another shot now that
I've dug into it more.

Hope that helps!

Damon

[outoftime]

Hey all,

Just came across this thread - following up on Damon's post, Sunspot
is designed to be pluggable with any data store you'd like to use it
with. All you have to do is define a couple of tiny adapter classes
that tell Sunspot how to pull data from the data store given a primary
key, and what the primary key of a given model is.

An example of an adapter is the ActiveRecord adapter that ships with
the Sunspot::Rails plugin:

http://github.com/outoftime/sunspot/blob/master/sunspot_rails/lib/sunspot/rails/adapters.rb

Note that the adapter has to be registered with Sunspot so that it
knows to use it (lines 4-5):

http://github.com/outoftime/sunspot/blob/master/sunspot_rails/rails/init.rb

Damon, if you'd be willing to share your MongoMapper adapter, I'd love
to see it - seems as if more and more people are looking to integrate
Sunspot and MongoMapper, and I'd love to get a package out so that
folks don't have to hand-roll their adapters.

Also, just to clarify, RSolr is fairly new and very actively
maintained; it's just a much lower-level API to Solr. Sunspot is
actually built on top of it.

Cheers,
Mat Brown

On Jan 27, 8:48 pm, Ericson Smith <esconsu...@gmail.com> wrote:
> We're using Ruby with MongoMapper and Solr.
>
> The rsolr gem is a bit older, but pretty neat. We tapped into before_create,
> after_update and after_delete mongomapper hooks to use rsolr to send update
> and delete calls over http to the solr server.
>
> Solr runs on a biggish server here, and in our dev environment, we're using
> it to pretty much show any screen that needs some kind of
> browse/filter/search/pagination, thus bypassing MongoDB entirely for any
> task not including create, update, delete and view (no list screens call
> upon mongo!).
>
> So we have pretty much real-time search with this setup (a couple of seconds
> at most delay for anything).
>
> It took us like an hour or two to write a simple solr class to do updates
> and deletes, and to hook it into our mongomapper models. We use rsolr to do
> searches too, which works out great.
>
> - Ericson Smith
> W3matter LLC

> CTO:http://www.funadvice.comOn Wed, Jan 27, 2010 at 8:29 PM, Al Cripps <c...@mtsu.edu> wrote:
> >  Damon,
> > Could you post additional information on how you did the integration and/or
> > provide additional information  else where or contact me at my email
> > address?  This is something that I have to do in the near future for a Rails
> > application and Sunspot :-)
> > thanks,
> > al
>
> > Damon P. Cortesi wrote:
>
> > Hi Ken,
>
> > I just did this over the weekend. Not sure what language you're using,
> > but I'm in ruby and ended up using the sunspot-solr library and
> > defining the appropriate adapters. As far as I know, sunspot-solr just
> > POSTs documents to the solr HTTP server with the appropriate XML.
> > Tapping into the replication protocol would be much better, but this
> > works OK for the moment.
>
> > Damon
>

> > On Jan 18, 2:20 pm, ken <kreis...@gmail.com> <kreis...@gmail.com> wrote:
>
> >  Has anyone recently been using Solr to index data in MongoDB?
> > If so, how are you configuring the Solr data-import handler to accept
> > the data?
>
> > If we can figure out a stable solution for this, then we might be
> > interested in switching from Postgres to MongoDB. The search doesn't
> > have to be real-time for us (though near real-time would be nice).
>
> > Cheers,
> > Ken
>
> >      --
> > You received this message because you are subscribed to the Google Groups
> > "mongodb-user" group.
> > To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>

[outoftime]

Eek, sorry - was looking at a cached view of the thread - thanks for
posting that, Damon!

Mat

On Jan 28, 10:41 am, outoftime <mat.a.br...@gmail.com> wrote:
> Hey all,
>
> Just came across this thread - following up on Damon's post, Sunspot
> is designed to be pluggable with any data store you'd like to use it
> with. All you have to do is define a couple of tiny adapter classes
> that tell Sunspot how to pull data from the data store given a primary
> key, and what the primary key of a given model is.
>
> An example of an adapter is the ActiveRecord adapter that ships with
> the Sunspot::Rails plugin:
>

> http://github.com/outoftime/sunspot/blob/master/sunspot_rails/lib/sun...

>
> Note that the adapter has to be registered with Sunspot so that it
> knows to use it (lines 4-5):
>

> http://github.com/outoftime/sunspot/blob/master/sunspot_rails/rails/i...

[mepatterson]

Ok, so I'm giving this a shot as a 10-minute "let's see how it does
vs. my homebrew sphinx setup with mongo". I can't seem to get it to
index anything. What am I missing?

I put a solr_config.rb in /lib that looks like the one Damon
presented. I put the following after the class declaration in my
Project model:

Sunspot.setup(Project) do
text :title, :boost => 2.0
text :summary
string :title
string :summary
time :created_at
boolean :completed, :using => :completed?
boost { completed? ? 2.0 : 1.0 }
end

Then I tried

Sunspot.search(Project) do
keywords 'truth'
end

I know for a fact that the word 'truth' is in both the title and
summary of several Projects. But no dice. No searches come up with
any results.

I figured maybe I need to reindex, so I tried:

Project.all.each {|p| p.save}

But that doesn't seem to index anything into solr, according to the
web-based tool:
searcherName : Searcher@de26e52 main
caching : true
numDocs : 0
maxDoc : 0

What big huge step am I missing here? Or is there some magic reindex
method that I need to call when using Mongo that's not documented?

On Jan 28, 10:08 am, outoftime <mat.a.br...@gmail.com> wrote:
> Eek, sorry - was looking at a cached view of the thread - thanks for
> posting that, Damon!
>
> Mat
>
> On Jan 28, 10:41 am, outoftime <mat.a.br...@gmail.com> wrote:
>
>
>
> > Hey all,
>
> > Just came across this thread - following up on Damon's post,Sunspot
> > is designed to be pluggable with any data store you'd like to use it
> > with. All you have to do is define a couple of tiny adapter classes

> > that tellSunspothow to pull data from the data store given a primary

> > key, and what the primary key of a given model is.
>
> > An example of an adapter is the ActiveRecord adapter that ships with

> > theSunspot::Rails plugin:
>
> >http://github.com/outoftime/sunspot/blob/master/sunspot_rails/lib/sun...
>
> > Note that the adapter has to be registered withSunspotso that it

> > knows to use it (lines 4-5):
>
> >http://github.com/outoftime/sunspot/blob/master/sunspot_rails/rails/i...
>
> > Damon, if you'd be willing to share your MongoMapper adapter, I'd love
> > to see it - seems as if more and more people are looking to integrate

> >Sunspotand MongoMapper, and I'd love to get a package out so that

> > > > application andSunspot:-)

> > > > thanks,
> > > > al
>
> > > > Damon P. Cortesi wrote:
>
> > > > Hi Ken,
>
> > > > I just did this over the weekend. Not sure what language you're using,

> > > > but I'm in ruby and ended up using thesunspot-solr library and
> > > > defining the appropriate adapters. As far as I know,sunspot-solr just

> > > > POSTs documents to the solr HTTP server with the appropriate XML.
> > > > Tapping into the replication protocol would be much better, but this
> > > > works OK for the moment.
>
> > > > Damon
>
> > > > On Jan 18, 2:20 pm, ken <kreis...@gmail.com> <kreis...@gmail.com> wrote:
>
> > > >  Has anyone recently been using Solr to index data in MongoDB?
> > > > If so, how are you configuring the Solr data-import handler to accept
> > > > the data?
>
> > > > If we can figure out a stable solution for this, then we might be
> > > > interested in switching from Postgres to MongoDB. The search doesn't
> > > > have to be real-time for us (though near real-time would be nice).
>
> > > > Cheers,
> > > > Ken
>
> > > >      --
> > > > You received this message because you are subscribed to the Google Groups
> > > > "mongodb-user" group.
> > > > To post to this group, send email to mongod...@googlegroups.com.
> > > > To unsubscribe from this group, send email to

> > > > mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>

[Ericson Smith]

Sometimes we try too much to use the various plugins that tie into active_record type installations.

I'd suggest that you simply use rsolr and tie it into an after_create call to send the saved document to your solr server. We did it like that, and the nice thing is that we understand exactly whats going on in all of our document save lifecycle.

For the model(s) we wanted indexed, we added a #model.to_xml method to create the xml needed the post, then in the after_create callback, we used rsolr to post that xml to solr. We also used a after_update callback to update search documents as well.

We also used rsolr in the after_destroy callback to remove items from the index. With the latest version, we're getting near-real-time indexing in the order of a half second or so.

It's super-important to play with your installed Solr installation at the URL level to make sure things are working as well.

- Ericson

To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

[mepatterson]

Okay, I figured this out. Figured I'd post for the benefit of those
that come after me.

I had to change the snippet Damon posted to look like this:

module MongoAdapter
class InstanceAdapter < Sunspot::Adapters::InstanceAdapter
def id
@instance.id
end
end

class DataAccessor < Sunspot::Adapters::DataAccessor
def load(id)
@clazz.find(Mongo::ObjectID.from_string(id.to_s))
end

def load_all(ids)
@clazz.find(ids.map{|i| Mongo::ObjectID.from_string(i.to_s)})
end
end
end

And then, I figured out how to do a full reindex:

Sunspot.index(Project.all)
Sunspot.commit

And that pretty much did it. solr now has documents and I intend to
start adding more stuff to what Damon started to make it a bit more
robust.

woot!

[Aníbal Rojas]

Ericson,

How many updates a minute are you pupmping into Solr? I am
interested in trying Solr integration with MongoDB, but in my scenario
I need to handle high update rates. Would be interesting knowing your
experience.

Best regards,

--
Aníbal Rojas
Ruby on Rails Web Developer
http://www.google.com/profiles/anibalrojas

[Ericson Smith]

Anibal,

Its not in production as yet, but we did some benchmarks recently. I was able to pump 10,000 records in around 5 minutes in a local network environment. Thats around 2000 each minute.

Our production environment would need to push updates per second, which is well below that.

- Ericson

2010/1/29 Aníbal Rojas <aniba...@gmail.com>

[Aníbal Rojas]

Ericson,

Thanks a lot! Sounds good, I was worried about Solr not being able
to keep up with a high update rate. What are you using or planning to
use? Cloud? Dedicated?

And thanks again,

--
Aníbal Rojas
Ruby on Rails Web Developer
http://www.google.com/profiles/anibalrojas

[Ericson Smith]

Anibal,

You're welcome.

We're currently using dedicated. But I've been playing with Amazon and Rackspace Cloud for some smaller sites. So no decision yet!

[tommy c]

Do you do a commit every time on object after_create, after_destroy?
If so, will this be problematic since numerous commits are being
issued?

On Jan 28, 4:25 pm, Ericson Smith <esconsu...@gmail.com> wrote:
> Sometimes we try too much to use the various plugins that tie into
> active_record type installations.
>
> I'd suggest that you simply use rsolr and tie it into an after_create call
> to send the saved document to your solr server. We did it like that, and the
> nice thing is that we understand exactly whats going on in all of our
> document save lifecycle.
>
> For the model(s) we wanted indexed, we added a #model.to_xml method to
> create the xml needed the post, then in the after_create callback, we used
> rsolr to post that xml to solr. We also used a after_update callback to
> update search documents as well.
>
> We also used rsolr in the after_destroy callback to remove items from the
> index. With the latest version, we're getting near-real-time indexing in the
> order of a half second or so.
>
> It's super-important to play with your installed Solr installation at the
> URL level to make sure things are working as well.
>
> - Ericson
>

> > > > > > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>

> > <mongodb-user%2Bunsubscribe@google groups.com>
> > > > > > .
> > > > > > For more options, visit this group at
> > > > > >http://groups.google.com/group/mongodb-user?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "mongodb-user" group.
> > To post to this group, send email to mongod...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > mongodb-user...@googlegroups.com<mongodb-user%2Bunsu...@googlegroups.com>

[Damon P. Cortesi]

I know this is not the approach the sunspot-rails library takes. They
state:
"rather than issuing a commit every time a document is added or
removed, Sunspot::Rails issues a commit at the end of any request
where data has been added to or removed from Solr"

I'm not 100% true what the exactly request lifecycle it's referring to
is, but I'm assuming it's just referring to an HTTP request.

Solr can also be configured to commit after a period of time since
last change, or based on number of documents added and this is a good
approach if your indexed data doesn't need to be immediately
accessible. See wiki.apache.org/solr/SolrConfigXml

Damon

On Jan 29, 8:17 am, tommy c <wheels...@gmail.com> wrote:
> Do you do a commit every time on object after_create, after_destroy?
> If so, will this be problematic since numerous commits are being
> issued?
>
> On Jan 28, 4:25 pm, Ericson Smith <esconsu...@gmail.com> wrote:
>
>
>
> > Sometimes we try too much to use the various plugins that tie into
> > active_record type installations.
>
> > I'd suggest that you simply use rsolr and tie it into an after_create call

> > to send the saved document to yoursolrserver. We did it like that, and the

> > nice thing is that we understand exactly whats going on in all of our
> > document save lifecycle.
>
> > For the model(s) we wanted indexed, we added a #model.to_xml method to
> > create the xml needed the post, then in the after_create callback, we used

> > rsolr to post that xml tosolr. We also used a after_update callback to

> > update search documents as well.
>
> > We also used rsolr in the after_destroy callback to remove items from the
> > index. With the latest version, we're getting near-real-time indexing in the
> > order of a half second or so.
>

> > It's super-important to play with your installedSolrinstallation at the

> > > But that doesn't seem to index anything intosolr, according to the

> > > > > >Solrruns on a biggish server here, and in our dev environment, we're

> > > using
> > > > > > it to pretty much show any screen that needs some kind of
> > > > > > browse/filter/search/pagination, thus bypassing MongoDB entirely for
> > > any
> > > > > > task not including create, update, delete and view (no list screens
> > > call
> > > > > > upon mongo!).
>
> > > > > > So we have pretty much real-time search with this setup (a couple of
> > > seconds
> > > > > > at most delay for anything).
>

> > > > > > It took us like an hour or two to write a simplesolrclass to do

> > > updates
> > > > > > and deletes, and to hook it into our mongomapper models. We use rsolr
> > > to do
> > > > > > searches too, which works out great.
>
> > > > > > - Ericson Smith
> > > > > > W3matter LLC
> > > > > > CTO:http://www.funadvice.comOnWed, Jan 27, 2010 at 8:29 PM, Al
> > > Cripps <c...@mtsu.edu> wrote:
> > > > > > >  Damon,
> > > > > > > Could you post additional information on how you did the
> > > integration and/or
> > > > > > > provide additional information  else where or contact me at my
> > > email
> > > > > > > address?  This is something that I have to do in the near future
> > > for a Rails
> > > > > > > application andSunspot:-)
> > > > > > > thanks,
> > > > > > > al
>
> > > > > > > Damon P. Cortesi wrote:
>
> > > > > > > Hi Ken,
>
> > > > > > > I just did this over the weekend. Not sure what language you're
> > > using,

> > > > > > > but I'm in ruby and ended up using thesunspot-solrlibrary and

> > > > > > > defining the appropriate adapters. As far as I know,sunspot-solr
> > > just

> > > > > > > POSTs documents to thesolrHTTP server with the appropriate XML.

> > > > > > > Tapping into the replication protocol would be much better, but
> > > this
> > > > > > > works OK for the moment.
>
> > > > > > > Damon
>
> > > > > > > On Jan 18, 2:20 pm, ken <kreis...@gmail.com> <kreis...@gmail.com>
> > > wrote:
>

> > > > > > >  Has anyone recently been usingSolrto index data in MongoDB?
> > > > > > > If so, how are you configuring theSolrdata-import handler to

> > > accept
> > > > > > > the data?
>
> > > > > > > If we can figure out a stable solution for this, then we might be
> > > > > > > interested in switching from Postgres to MongoDB. The search
> > > doesn't
> > > > > > > have to be real-time for us (though near real-time would be nice).
>
> > > > > > > Cheers,
> > > > > > > Ken
>
> > > > > > >      --
> > > > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > > > "mongodb-user" group.
> > > > > > > To post to this group, send email to mongod...@googlegroups.com
> > > .
> > > > > > > To unsubscribe from this group, send email to

> > > > > > > mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>

> > > <mongodb-user%2Bunsubscribe@google groups.com>
> > > > > > > .
> > > > > > > For more options, visit this group at
> > > > > > >http://groups.google.com/group/mongodb-user?hl=en.
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "mongodb-user" group.
> > > To post to this group, send email to mongod...@googlegroups.com.
> > > To unsubscribe from this group, send email to

> > > mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>

[smnirven]

I've got a question regarding setting up the sunspot index on multiple
models. Basically I have a location document, and a property
document, and I would like to build indexes and be able to perform
searches on either model. Here is my setup code:

# Hook our MongoMapper model into Solr
require 'sunspot'

module MongoAdapter
class InstanceAdapter < Sunspot::Adapters::InstanceAdapter
def id
@instance.id
end
end

class DataAccessor < Sunspot::Adapters::DataAccessor
def load(id)

@clazz.find(id)
end
end
end

Sunspot::Adapters::DataAccessor.register(MongoAdapter::DataAccessor,
Location, Property)
Sunspot::Adapters::InstanceAdapter.register(MongoAdapter::InstanceAdapter,
Location, Property)

Sunspot.setup(Location) do
text :name
string :name_partial do
name.downcase
end
end

Sunspot.setup(Property) do
text :name
string :name_partial do
name.downcase
end
end

I get no errors when my application starts and executes this code,
however, when I try to perform a search on the location model, I get a
Sunspot::NoAdapterError exception. Here is my search code:

s = Sunspot.search(Location) do
with(:name_partial).starting_with(query_string.downcase)
end

s.results.inspect

And the error I get, which happens when I call s.results:

Sunspot::NoAdapterError (No adapter is configured for NilClass or its
superclasses. See the documentation for Sunspot::Adapters):

Any ideas?

Thanks

-Tom

On Jan 30, 3:20 pm, "Damon P. Cortesi" <d.lifehac...@gmail.com> wrote:
> I know this is not the approach thesunspot-rails library takes. They

> state:
> "rather than issuing a commit every time a document is added or

> removed,Sunspot::Rails issues a commit at the end of any request

> > > > > > > browse/filter/search/pagination, thus bypassingMongoDBentirely for

> > > > > > > > interested in switching from Postgres toMongoDB. The search

> > > > doesn't
> > > > > > > > have to be real-time for us (though near real-time would be nice).
>
> > > > > > > > Cheers,
> > > > > > > > Ken
>
> > > > > > > >      --
> > > > > > > > You received this message because you are subscribed to the Google
> > > > Groups
> > > > > > > > "mongodb-user" group.

> > > > > > > > To post to this group, send email tomongo...@googlegroups.com

> > > > .
> > > > > > > > To unsubscribe from this group, send email to
> > > > > > > >mongodb-user...@googlegroups.com<mongodb-user%2Bunsubscribe@google groups.com>
> > > > <mongodb-user%2Bunsubscribe@google groups.com>
> > > > > > > > .
> > > > > > > > For more options, visit this group at
> > > > > > > >http://groups.google.com/group/mongodb-user?hl=en.
>
> > > > --
> > > > You received this message because you are subscribed to the Google Groups
> > > > "mongodb-user" group.

> > > > To post to this group, send email tomongo...@googlegroups.com.

[outoftime]

Tom - I might be able to help you out with this - can you post the
full stack trace to ruby-s...@googlegroups.com ?

Thanks!
Mat

> ...
>
> read more »

[smnirven]

Turns out the problem was that my database and index were out of
sync. I didn't realize this, but when you run the index, it's
additive, and doesn't remove keys for data that is no longer in the
database. To solve this problem, I had to run these commands to
rebuild my index for the location table:

Sunspot.remove_all(Location)
Sunspot.commit
Sunspot.index(Location.all)
Sunspot.commit

Is there a better way of doing this? Thanks for the help.

-Tom

On Feb 24, 7:58 am, outoftime <mat.a.br...@gmail.com> wrote:
> Tom - I might be able to help you out with this - can you post the
> full stack trace to ruby-s...@googlegroups.com ?
>
> Thanks!
> Mat
>
> On Feb 23, 10:08 am, smnirven <smnir...@gmail.com> wrote:
>

> > I've got a question regarding setting up thesunspotindex on multiple

> > s =Sunspot.search(Location) do

> >   with(:name_partial).starting_with(query_string.downcase)
> > end
>
> > s.results.inspect
>
> > And the error I get, which happens when I call s.results:
>
> >Sunspot::NoAdapterError (No adapter is configured for NilClass or its

> > superclasses. See the documentation forSunspot::Adapters):

> ...
>
> read more »

[mwaschkowski]

Hi,

There is a lot of functionality overlap between Mongo and Solr, what
is the underlying motivations for hooking them up?

Thanks,

Mark

[Howard van Rooijen]

I'm also very interested in the combination of either MongoDB & Solr / Elastic Search (both of which are based on Lucene). I'm very interested in the full text search, faceted search, term vector, synonyms & stemming support in Solr / Elastic Search. 

Elastic Search, although newer and less proven looks like an interesting fit as it seems to be a hybrid between a search engine and a document database - but also uses JSON at it's core - which means integration with MongoDB could be quite simple - whereas Solr still uses XML - this would require some form of translation / adapter layer. 

I'm a .NET Developer - and have been looking at the C# Driver and wondering if some kind of interception framework could be added that would allow extensibility points during CRUD operations which could then be mirrored to a Solr / Elastic Search Server. One of the first problems that I could think of was that operations like UpdateAll don't seem to return a list of Ids of documents affected - thus there is no-way to re-query and reflect these changes to the Solr / ES Server.

I've only been looking at MongoDB for a week - so I'm not totally up to speed with it's architecture & extensibility points - but would the driver be the correct place for this type of feature - or is there something built into MongoDB that would offer a better solution?

I recently built an ecommerce platform that was based on SQL Server + Solr and would be very interested in seeing if a similar architecture could be built with MongoDB + Solr as the more I learn about document databases - the more I think they could ease much of the pain dealing with retailers product / SKU schema changes.

Regards,

Howard

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.

To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.

[Mathias Stearn]

the best way to react to data modifications in mongo is to watch the
oplog. It is used for replication so you can get a full description of
every change: http://www.mongodb.org/display/DOCS/Replication+Internals

[go canal]

Hi,

For a simple reason: I need to support Office and PDF full-text search. Solr uses Tika to do the text extraction. If I store the files in MongoDB, how do I do this ?
 

rgds,
canal

---

From: mwaschkowski <mwasch...@gmail.com>
To: mongodb-user <mongod...@googlegroups.com>
Sent: Fri, February 26, 2010 11:31:55 PM
Subject: [mongodb-user] Re: Integrating MongoDB with Solr

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.

To unsubscribe from this group, send email to mongodb-user+unsub...@googlegroups.com.

[mwaschkowski]

Right. OK, just wanted to make sure I wasn't missing something.

btw - mongo does to full text search:

http://www.mongodb.org/display/DOCS/Full+Text+Search+in+Mongo

so if you used tika you could do full text search by keeping an extra
field with the full text of each doc and then do full text queries
against that. I know that its not as advanced as solr, but might be
simpler to setup.

One other thing to consider, mongo is supposed to (see roadmap) get
fuller (haha) full text search capabilities in 2010 Q2, which is
relatively soon, so you may want to wait...

Best,

Mark

On Mar 6, 10:50 pm, go canal <goca...@yahoo.com> wrote:
> Hi,
> For a simple reason: I need to support Office and PDF full-text search. Solr uses Tika to do the text extraction. If I store the files in MongoDB, how do I do this ?
>  rgds,
> canal
>
> ________________________________

> From: mwaschkowski <mwaschkow...@gmail.com>

> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.- Hide quoted text -
>
> - Show quoted text -

[go canal]

Indeed, a fuller full-text search is what I need. In fact, this is likely the last feature that I am waiting for before I stop evaluating CouchDB, which has a good integration with Lucene today.

Yes, I am aware of the current search feature, but I am too lazy to add additional code into my application to handle the extraction, highlight, scoring, etc. 

I can not wait for the new solution !

Thanks for your info,

canal

---

From: mwaschkowski <mwasch...@gmail.com>
To: mongodb-user <mongod...@googlegroups.com>

Sent: Mon, March 8, 2010 9:23:43 AM

> To unsubscribe from this group, send email to mongodb-user+unsub...@googlegroups.com.

> For more options, visit this group athttp://groups.google.com/group/mongodb-user?hl=en.- Hide quoted text -
>
> - Show quoted text -

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.

To unsubscribe from this group, send email to mongodb-user+unsub...@googlegroups.com.

[tommy c]

I found out i need a data import handler from mongoDB to solr as well.
I put up a reward for anyone who code up this little project
http://nextsprocket.com/tasks/create-a-solr-data-import-handler-for-mongodb