GeoNear query not retreiving document

306 views
Skip to first unread message

Gustavo Campos

unread,
Oct 22, 2016, 10:04:04 PM10/22/16
to mongodb-user

Hi all, hope you are well.

I have a collection with a few documents and when I run a specific geoNear query one of my objects is not retrieved as it should. This only happens with this specific document and only with this specific query (that I detected so far). I removed all other documents from the collection that I could to still replicate the error, which means that if you remove ANY of the objects from the collection the query will work as expected.

Here are the collection's documents: Collection

To replicate the error after you create the collection and add the documents to it, you need to create a 2dsphere index:


db
.collectionName.createIndex( { loc : "2dsphere" } )

And run the following command:

db.collectionName.aggregate([
  {"$geoNear":{"near":{"type":"Point","coordinates":[144.960662,-37.818133]},"spherical":true,"distanceField":"dist","limit":null,"maxDistance":5000,"query":{"hobby_id":{"$in":[1,2,7]}}}},
  {"$sort":{"datetime":-1}}
])

The problematic document is the following:

{
    "_id" : ObjectId("57f47e690e8620049e3e4511"),
    "loc" : {
        "type" : "Point",
        "coordinates" : [ 
            145.001541137695, 
            -37.8015441894531
        ]
    },
    "datetime" : ISODate("2016-10-05T15:15:37.000+11:00"),
    "msg" : "57f47e690e8620049e3e4511",
    "filter_id" : 1.0
}

Note that is was supposed to be the second element when you run the query above, but it is not shown at all. Its distance to the given point is 4041.74170931192 meters so it's not being filtered out by the geoNear distance filter.

If you delete any other document from the collection it will be the second item shown and if you remove the filter_id 2 or 7 from the query it will also work and the document will show up as the second in line.

The MongoDB version I'm using is 3.2.7 and it is running on a Ubuntu 14.04. Currently, this is the only document with which I've been able to see this behavior but I guess it's essential to understand what is causing it and fix it before it happens in production. Does anyone have a clue on what might be going on? It looks like a MongoDB bug to me. I'll be pleased to provide any missing information upon request.

Thank you all very much in advance!

Stephen Steneker

unread,
Oct 23, 2016, 4:09:12 PM10/23/16
to mongodb-user
On Sunday, 23 October 2016 13:04:04 UTC+11, Gustavo Campos wrote:

The problematic document is the following:

{
    "_id" : ObjectId("57f47e690e8620049e3e4511"),
    "loc" : {
        "type" : "Point",
        "coordinates" : [ 
            145.001541137695, 
            -37.8015441894531
        ]
    },
    "datetime" : ISODate("2016-10-05T15:15:37.000+11:00"),
    "msg" : "57f47e690e8620049e3e4511",
    "filter_id" : 1.0
}

Note that is was supposed to be the second element when you run the query above, but it is not shown at all. Its distance to the given point is 4041.74170931192 meters so it's not being filtered out by the geoNear distance filter.

Hi Gustavo,

If your aggregation query and sample document are as provided, this document won't be found because it doesn't match your query filter ("query":{"hobby_id":{"$in":[1,2,7]}})

The document does not include a hobby_id field; if I remove the query filter from your aggregation query the document is returned.

Regards,
Stephen

Message has been deleted

Gustavo Campos

unread,
Oct 23, 2016, 6:24:32 PM10/23/16
to mongodb-user
Hi Stephen, thank you very much for taking your time to answer me. This was actually a mistake I did when removing unnecessary fields and changing some dummy names in order to post the question here. The correct query would be:
db.collectionName.aggregate([
{"$geoNear":{"near":{"type":"Point","coordinates":[144.960662,-37.818133]},"spherical":true,"distanceField":"dist","limit":null,"maxDistance":4500,"query":{"filter_id":{"$in":[1,2,7]}}}},
{"$sort":{"datetime":-1}}])

And the problem still occurs even though the document, as far as I can see, matches all the requirements.

Gustavo Campos

unread,
Oct 27, 2016, 8:10:57 PM10/27/16
to mongodb-user
anyone???

Wan Bachtiar

unread,
Nov 2, 2016, 11:29:37 PM11/2/16
to mongodb-user

If you delete any other document from the collection it will be the second item shown and if you remove the filter_id 2 or 7 from the query it will also work and the document will show up as the second in line.

Hi Gustavo,

This is because if you specify filter_id for value 1, 2 or 7 you have the aggregation result over 100 documents. As you specify limit to null, $geoNear operator uses the default limit of 100 documents.

Essentially, you are getting the first 100 documents that match query and whose coordinates satisfy maxDistance, sorting those 100 documents by datetime. This is why if you remove filter_id 2 or 7 you are getting the result, because the total aggregation result is less than 100 documents.

You can try this, by specifying a limit to 105, for example:

db.collectionName.aggregate([ 
      {"$geoNear":
         {"near":
             {"type":"Point",
             "coordinates":[144.960662,-37.818133]},
             "spherical":true,
              "distanceField":"dist"
,
              "limit":105,
              "maxDistance":4500,
              "query":{"filter_id":{"$in":[1,2,7]}}
         }}, 
      {"$sort":{"datetime":-1}}
]);

Regards,

Wan.

Gustavo Campos

unread,
Nov 2, 2016, 11:40:53 PM11/2/16
to mongodb-user
Hi Wan,

Thank you very much for your help, I really appreciate it. I was under the impression that setting the value to null I would be actually setting it to "infinite", which means it would return all the documents that match the query and the distance. Is there a way of doing this?

Wan Bachtiar

unread,
Nov 3, 2016, 3:26:01 AM11/3/16
to mongodb-user

it would return all the documents that match the query and the distance. Is there a way of doing this?

Hi Gustavo,

$geoNear is an operation that outputs documents in order of nearest to farthest from a specified point. In other words, the output returned to you is already ordered by distance. Setting an unlimited result sorted may not be ideal in this case.

An alternative is to use $geoWithin instead, which outputs documents that exist entirely within a specified shape. Using your example:

db.collectionName.find({loc:
                        {$geoWithin:
                             {$centerSphere:[[144.960662,-37.818133], 5000]}
                        }, 
                        filter_id:{$in:[1,2,7]}
});

See also $centerSphere geo shape operator. The example query above was written for MongoDB v3.2.

For more geospatial query operators see GeoSpatial Query Operators. You may also find Find Restaurants with GeoSpatial Queries a useful resource.

Regards,

Wan.

Gustavo Campos

unread,
Nov 6, 2016, 6:54:17 PM11/6/16
to mongodb-user
Hi Wan,

Once more, thank you very much for the enlightening answer. It became clear to me that GeoWithin would be the best option for the kind of filtering I want to accomplish. With that said, the GeoWithin query creates another problem for me, which is the fact that it does not output the distance between the query coordinates and the document as we have using GeoNear. I could calculate this distance on the client side but for that, I'd have to reveal the coordinates, which is a security issue for this particular application. Hence, it is mandatory that the distance is calculated on the server side and iterating through the results and doing the math manually seems like a very poor choice, do you know of an alternative that would fit this specific situation better?

Best regards,

Gustavo

Wan Bachtiar

unread,
Nov 7, 2016, 10:26:19 PM11/7/16
to mongodb-user

 Hence, it is mandatory that the distance is calculated on the server side and iterating through the results and doing the math manually seems like a very poor choice, do you know of an alternative that would fit this specific situation better?

Hi Gustavo,

If you require the distance being calculated server side, then $geoNear would be a good operator for it. Depending on the application use case, you could set a reasonably high number for your requirement. For example, displaying 1000 nearest objects on a map could be sufficient.

Another option is to count the number of results using $geoWithin , then use the result to limit $geoNear. For example, if the count returns 237 items, then use the number as limit. Although depending on your application requirements and use case, running two queries may not be ideal.

Regards,

Wan.


Reply all
Reply to author
Forward
0 new messages