full text search (mongodb 2.4)

[Serkan Durusoy]

Is it possible to utilize mongodb's new full text search capabilities from within meteor?

In fact, has anyone solved the search problem in any way? (solr, elasticsearch etc)

What I'm wondering is:

a) Can I add full text search to a meteor application

b) Can I make it reactive eg a list of fresh results get displayed at each keystroke in the search field

[wars...@gmail.com]

Also very interested in anyone who has solved search particularly with lucene/elastic search or solr. 

[Dan Shields]

(b) would be a matter of simply calling your server's search method again on keyup events in your search field. 

(a) is something that I'm also interested in. I suspect it would involve some private API for now, to get the native MongoDB collection/database handle. I might dig around in the source code to see if I can find anything helpful.

[Sacha Greif]

Did anybody make any progress on this? I'm looking into implementing search in Telescope and I'd be interested to know if anybody got it to work in their own app?

[Ry Walker]

I'm investigating full-text options too for a project we're working on.

A) Seems Atmosphere is searching multiple fields... see:

https://github.com/oortcloud/atmosphere/blob/master/app/client/display.js

Template.packages.packages = function() {

  keywords = new RegExp(Session.get("search_keywords"), "i");

  return Packages.find({$or:[{name:keywords},{description:keywords}]}, {sort: {'updatedAt': -1}});

};

B) Another idea suggested to me by Modulus tonight — http://dev.searchbox.io/nodejs/ has a full text service.

-Ry

--
You received this message because you are subscribed to the Google Groups "meteor-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meteor-talk...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

-Ry

Ry Walker | Partner at Differential | 513.417.2163 | @rywalker | LinkedIn

[Thimo Brinkmann]

Hey Serkan,

I introduced this functionality in my app when MongoDB 2.4 came out using the "native" MongoDB feature.

• Make sure to enable text search (via config file or run a command on startup, this is well explained in the mongodb manual)
• Create text index

I'm using bootstrap 2.x typeahead to get updates after each keystroke. I call my method "getSuggestionsForSearchTermV2". My V2 is based on MongoDB text search feature, while V1 was based on Meteor.Collection.find() method + RegEx

Code snippet UI-Component (bootstrap required):

Template.navigation.rendered = function () {

$('#searchfield').typeahead({

items: 11,

minLength: 4,

source: function (query, process) {

Meteor.call("getSuggestionsForSearchTermV2", ".*" + query.replace(/([.?*+^$[\]\\(){}|-])/g, "\\$1") + ".*", function (error, result) {

if (result && result.length) {

result.unshift(query.trim());

}

process(result);

});

});

}

Please be aware that bootstrap 3 will drop this typeahead in favor of jquery typeahead component.

On the server side, this is the relevant part of the method call:

getSuggestionsForSearchTermV2: function (searchterm) {

var searchterm_mod = '';

var searchterms = searchterm.trim().split(" ");

for (var i = 0; i < searchterms.length; i++) {

searchterm_mod+= '\"' + searchterms[i] + '\"' + ' ';

}

searchterm_mod = searchterm_mod.replace(/\.\*|\(|\)/g,"").trim();

Future = Npm.require('fibers/future');

var fut = new Future();

Meteor._RemoteCollectionDriver.mongo.db.executeDbCommand({"text":"<collectionname>",search: searchterm_mod, limit:10, project:{_id:0, name:1}},

function (error,results){

if (results && results.documents[0].ok === 1)

{

var ret = results.documents[0].results;

fut.ret(_.uniq(_.map(_.pluck(ret, 'obj'), function (text) {

return text.name;

})));

}

}

);

return fut.wait();

}

As you can see, you need to run the executeCommand() method, which is not very well known and not documented in the mongo nodejs driver.

Of course the MongoDB text search is still beta and has many limitations, but it works quite well for my needs. If you want to see it live send me a message, I will send you a link to my private application so you can play around with it. Also, it might be not the best to use the Meteor db handle like this, but I found no other way...

Replace text.name by the field you want to be returned as visible text to the user searching. You can leave out the projection of course if you need more information. The projection parameter is also well-documented in the MongoDB documentation.

Hope this helps you...

Thimo

[Sacha Greif]

Thanks, that's a big help! I actually asked a question on SO about just this the other day. 

Can I ask what you're doing with the result? Is your app's code publicly available by any chance?

[Sacha Greif]

Actually, I'm curious to know how you managed to enable text search in the first place? Is there a way to do it programatically? (with executeDbCommand() maybe?)

[Julien Chaumond]

I don't think that's possible – I don't think you can turn text-search on on your mongod instance at runtime.

Julien

[Sacha Greif]

Well, you can from the mongo console with "db.admin.runCommand({setParameter: 1, textSearchEnabled: true})". But it only works on the "admin" db, not on "meteor".

Alternatively, how would you turn it on permanently before runtime in a Meteor app?

--
You received this message because you are subscribed to a topic in the Google Groups "meteor-talk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/meteor-talk/x9kYnO52Btg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to meteor-talk...@googlegroups.com.

[Arunoda Susiripala]

Hi i have a plan to integrate mongo full text search into meteor. 

But not this week, possibly next week on MeteorHacks. 

--

You received this message because you are subscribed to a topic in the Google Groups "meteor-talk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/meteor-talk/x9kYnO52Btg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to meteor-talk...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
You received this message because you are subscribed to the Google Groups "meteor-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meteor-talk...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Arunoda Susiripala

@arunoda

https://github.com/arunoda

http://www.linkedin.com/in/arunoda

[Gabriel Pugliese]

Have you seen mongo-extensions ? https://github.com/jhoxray/meteor-mongo-extensions

Maybe it's possible the same way...

Gabriel Pugliese @gabrielsapo

[Arunoda Susiripala]

Yes. that could work. With some integration of permissions. 

[Thimo Brinkmann]

Hey Sacha,

to enable it permanently when using the meteor built-in mongodb instance, I had to change the code base and add the config parameter.

Have a look in the mongo_runner.js file, at the very bottom. You have this part:

    var proc = child_process.spawn(mongod_path, [

      '--bind_ip', '127.0.0.1',

      '--smallfiles',

      '--port', port,

      '--dbpath', data_path

    ]);

Which basically runs a new mongod instance. Just add a '--setparameter',<key=value> call here. Have a look at http://docs.mongodb.org/manual/reference/program/mongod/#cmdoption-mongod--setParameter

If you have your own standalone instance of MongoDB, that is of course no problem. In your mongodb.conf, just add this line:

setParameter = textSearchEnabled=true

[Lloyd Henning]

Thanks to the information in this thread, I've written up how I got mongodb text search working as an answer to Sasha's question on StackOverflow

http://stackoverflow.com/a/18258688/418782

Tempted to look more into a solr solution as I'm that happy with the mongodb text search for things like partial matches (in fact I've gone for just regexp on the project I'm doing).

Thanks

Lloyd

[Sacha Greif]

Thanks a lot for this!

[Thimo Brinkmann]

If anybody is still using the RemoteCollectionDriver approach, they just moved it to the MongoInternals object in 0.6.5 (Andrew Wilcox already discovered it).

MongoInternals.defaultRemoteCollectionDriver().mongo.db.executeDbCommand

instead of the previous

Meteor._RemoteCollectionDriver.mongo.db.executeDbCommand

[Stephan Tual]

Thank you ever so much Thimo, very useful!

[Gabriel Pugliese]

Question: What could be happening if the text search always returns [] ?

> db.channels.runCommand('text', {search: 'T'});

{

"queryDebugString" : "t||||||",

"language" : "english",

"results" : [ ],

"stats" : {

"nscanned" : 0,

"nscannedObjects" : 0,

"n" : 0,

"nfound" : 0,

"timeMicros" : 118

},

"ok" : 1

}

What are those strange chars within the queryDebugString ?

[Gabriel Pugliese]

The config I've taken from Meteorpedia:

Meteor.startup(function () {

    search_index_name = 'channels_index'

 

    // Remove old indexes as you can only have one text index and if you add 

    // more fields to your index then you will need to recreate it.

    Channels._dropIndex(search_index_name);

 

    Channels._ensureIndex({

        title: 'text'

    }, {

        name: 'channels_search'

    });

});

Gabriel Pugliese CodersTV.com @gabrielsapo

--

You received this message because you are subscribed to the Google Groups "meteor-talk" group.

To unsubscribe from this group and stop receiving emails from it, send an email to meteor-talk...@googlegroups.com.

[Dirk Porsche]

Hi,

I made a native Meteor package (no meteorite, yet) to enable full-text search. You might want to check it out:

It's on GitHub:

https://github.com/Crenshinibon/spomet

I have written a lengthy (beginners) tutorial on how to integrate Spomet into an application:

http://shiggyenterprises.wordpress.com/2013/09/28/developing-a-full-text-search-enabled-meteor-app/

Let me hear what you think.

Regards 

Dirk 

Am Dienstag, 14. Mai 2013 13:47:04 UTC+2 schrieb Serkan Durusoy: