Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
mongodb-user
Home
+ new post
About this group
Join this group
Message from discussion Do $or statements in the query parameter of MapReduce use the index?
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Nathan Hoad  
View profile  
 More options Sep 12 2011, 7:19 pm
From: Nathan Hoad <nat...@getoffmalawn.com>
Date: Mon, 12 Sep 2011 16:19:46 -0700 (PDT)
Local: Mon, Sep 12 2011 7:19 pm
Subject: Re: Do $or statements in the query parameter of MapReduce use the index?
Print | View thread | Show original | Report this message | Find messages by this author
To be clear, that was for a collection with 200 million records. I
checked the disk utilization with iostat during the query, and it was
a constant 98%, but performing the individual queries had very little
utilization. This is what gave me the impression that the index wasn't
being used.

I've tried using $in and it's much more manageable, taking exactly the
amount of time it should. Thanks!

However, I've compared the individual results to $in as you suggested,
and the output is different.

Individual:
{ "_id" : "1743", "value" : { "total_data" : 58016271 } }
{ "_id" : "23", "value" : { "total_data" : 103653535 } }

Using $in:
{ "_id" : "1743", "value" : { "total_data" : 58016271 } }
{ "_id" : "23", "value" : { "total_data" : 103653535 } }

The output of the map reduce call indicates that the exact same amount
of records were read and inputted for both types of queries, yet the
amounts are incorrect. What could cause this? Obviously my reduce has
a bug, but it's so simple I have no idea what could be causing this.

On Sep 10, 1:03 am, Kyle Banker <kyleban...@gmail.com> wrote:

> I don't believe that a difference of 45 minutes could possibly be
> explained by whether $or is using the index or not.

> For the record, the $or does use an index, but you should actually
> express this differently:

> query: { time: {'$gte': start, '$lte': end}, username: { $in: ['1743',
> '23'] } }

> I also suggest that you run the both the $orqueryand the individual
> queries separate to ensure that they both return the same total number
> of documents.

> On Sep 8, 9:09 pm, Nathan Hoad <nat...@getoffmalawn.com> wrote:

> > I have map and reduce functions defined as follows;

> > map = function() {
> >     emit(this.username, { total_data: this.sent + this.received });

> > }

> > reduce = function (key, values) {
> >     var result = {total_data:0};
> >     values.forEach(function (value) {result.total_data +=
> > value.total_data;});
> >     return result;

> > }

> > And I'm running the followingquery;

> > db.collection.mapReduce(m, r, {out:'myoutput',query: { time: {'$gte':
> > start, '$lte': end}, '$or': [ { username: '1743'}, {username:
> > '23'}] }, sort: {username: -1} });

> > If I run thequerywithout the $or, and do two separate queries for
> > the username, then the queries are reasonably fast (finish between 1
> > second and 2-3 minutes. But if I run it with the or statement, it
> > takes 45-50 minutes. I'm not very experienced with MongoDB so I'm
> > unsure if there's anything I can do to debug this, or figure out if/
> > why it's not using the index.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
©2013 Google