Map/reduce runs in parallel/distributed?
542 views
Skip to first unread message
Stodge
Feb 17, 2011, 8:22:58 PM2/17/11
to mongodb-user
So mongodb map/reduce isn't run in parallel across multiple nodes? I'm
sure the OReilly book says otherwise. Must dig it out and double
check. Except it's at work.
sure the OReilly book says otherwise. Must dig it out and double
check. Except it's at work.
Eliot Horowitz
Feb 17, 2011, 8:24:03 PM2/17/11
to mongod...@googlegroups.com
It is run in parallel across sharded nodes.
> --
> You received this message because you are subscribed to the Google Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
>
>
Stodge
Feb 17, 2011, 9:30:37 PM2/17/11
to mongodb-user
But not in a unsharded replica set?
Thanks
Mike
Thanks
Mike
Nat
Feb 17, 2011, 9:33:38 PM2/17/11
to mongod...@googlegroups.com
Nope. It won't
Luke Ehresman
Feb 17, 2011, 9:55:06 PM2/17/11
to mongod...@googlegroups.com
A map/reduce job cannot have multiple threads on the same node. But the whole point of map reduce is to run parallel on multiple nodes. So the same job is run simultaneously on multiple nodes, but only in one thread on any given node.
Luke
On Thu, Feb 17, 2011 at 8:22 PM, Stodge <sto...@gmail.com> wrote:
Andrew Armstrong
Feb 17, 2011, 10:28:17 PM2/17/11
to mongodb-user
It's definitely not as powerful as it can be (single threaded per
physical node).
I believe theres a Jira issue to make the javascript engine in use
multithreaded so that multiple cores can do mapreduce jobs sometime in
the future.
It would be cool also to see replica slaves possibly handle some of
the workload (slaveOk?).
On Feb 18, 1:55 pm, Luke Ehresman <l...@ehresman.org> wrote:
> A map/reduce job cannot have multiple threads on the same node. But the
> whole point of map reduce is to run parallel on multiple nodes. So the same
> job is run simultaneously on multiple nodes, but only in one thread on any
> given node.
>
> Luke
>
> On Thu, Feb 17, 2011 at 8:22 PM, Stodge <sto...@gmail.com> wrote:
> > So mongodb map/reduce isn't run in parallel across multiple nodes? I'm
> > sure the OReilly book says otherwise. Must dig it out and double
> > check. Except it's at work.
>
> --
> Luke Ehresman, l...@ehresman.org
> <l...@ehresman.org>CopperEgg -http://copperegg.com
> Tebros Systems -http://tebros.com
physical node).
I believe theres a Jira issue to make the javascript engine in use
multithreaded so that multiple cores can do mapreduce jobs sometime in
the future.
It would be cool also to see replica slaves possibly handle some of
the workload (slaveOk?).
On Feb 18, 1:55 pm, Luke Ehresman <l...@ehresman.org> wrote:
> A map/reduce job cannot have multiple threads on the same node. But the
> whole point of map reduce is to run parallel on multiple nodes. So the same
> job is run simultaneously on multiple nodes, but only in one thread on any
> given node.
>
> Luke
>
> On Thu, Feb 17, 2011 at 8:22 PM, Stodge <sto...@gmail.com> wrote:
> > So mongodb map/reduce isn't run in parallel across multiple nodes? I'm
> > sure the OReilly book says otherwise. Must dig it out and double
> > check. Except it's at work.
>
> --
> <l...@ehresman.org>CopperEgg -http://copperegg.com
> Tebros Systems -http://tebros.com
Adrien Mogenet
Feb 18, 2011, 2:12:12 AM2/18/11
to mongodb-user
Mongo team is aware that Map/Reduce might be the slowest operation on
MongoDB and there are several leads of optimization :)
I also think that using the new covered index in map/reduce jobs (when
possible) could be powerful.
MongoDB and there are several leads of optimization :)
I also think that using the new covered index in map/reduce jobs (when
possible) could be powerful.
Nat
Feb 18, 2011, 2:30:17 AM2/18/11
to mongodb-user
Reply all
Reply to author
Forward
0 new messages