Using external JavaScript libraries in Map/Reduce

[Stephen]

Hi,

We are in the process of writing some fairly complex map/reduce jobs
on our mongo DB. We would like to use some external libraries to make
our code more manageable (underscore.js right now but possibly more in
the future).

Is there a recommended way to import large js libraries into a mongo
DB. I know you can use db.system.js.insert to store javascript
snippets into variables. I started going down this route but noticed
the strings I pass it get cut off after a certain number of
characters.

So my questions are:
1) Has anybody done something similar and if so, how?
2) Can I call db.system.js.insert and provide a file name instead of a
string containing the javascript?
3) Is db.system.js.insert the best way to do this?

Thanks in advance,
Stephen

[Eliot Horowitz]

system.js is the best way.
The only cap should be 16mb (in 1.8)

> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>

[Andy Gregorowicz]

On Apr 11, 6:50 am, Stephen <stephen...@gmail.com> wrote:
> Hi,
>
> We are in the process of writing some fairly complex map/reduce jobs
> on our mongo DB.  We would like to use some external libraries to make
> our code more manageable (underscore.js right now but possibly more in
> the future).
>
> Is there a recommended way to import large js libraries into a mongo
> DB.  I know you can use db.system.js.insert to store javascript
> snippets into variables.  I started going down this route but noticed
> the strings I pass it get cut off after a certain number of
> characters.
>
> So my questions are:
> 1) Has anybody done something similar and if so, how?

One thing you can do is eval() the library on the DB connection. The
library will only be around for the duration of the connection, but
will be available for use in Map Reduce or other server side things. I
wrote a quick blog post about it here:

http://gregorowicz.blogspot.com/2010/12/using-underscorejs-with-mongodb.html

One major issue with this approach is that it doesn't allow you to do
sharding.

Another approach it to drop the framework right into the MapReduce
code. On a project I am working on, we have our map functions stored
in js files. They actually are templates (ERB in our case because we
are working in Ruby). So when we want to run a MapReduce job, we load
the template file, render it (which inserts Underscore.js right into
the map function) and then send the rendered map function to MongoDB.
This is kind of inefficient, since Underscore is getting re-evaluated
with every M/R. job.

I think it would be nice if JavaScript libraries coule be stored
server side and have opened up a feature request here:

https://jira.mongodb.org/browse/SERVER-2936

~Andy