geospatial search questions

[Lincoln Hochberg]

Hey guys,

Thanks very much for 1.3.3 - it's awesome!

I'm having a little trouble with geospatial search, which I'm testing.

I created the following index:

{

  "name" : "location.gps_",

  "ns" : "test.events",

  "key" : {

    "location.gps" : "2d"

  }

}

However, when I try to use it with a query like this:

{ "location.gps" : { "$near" : { "lat" : 40.719745 , "lon" : -73.955041}} , "_id" : { "$nin" : [ "4b8acc6e35b304d9dbdf6d90" , "4b8c244d468e04d925c6b191" ]} , "state" : "pre"}

I get the following error from the java driver:

java.lang.RuntimeException: db error [can't find index starting point]

    at com.mongodb.DBApiLayer$MyCollection.find(DBApiLayer.java:321)

    at com.mongodb.DBCursor._check(DBCursor.java:253)

    at com.mongodb.DBCursor._hasNext(DBCursor.java:374)

    at com.mongodb.DBCursor._fill(DBCursor.java:442)

    at com.mongodb.DBCursor.toArray(DBCursor.java:472)

    at com.mongodb.DBCursor.toArray(DBCursor.java:461)

    at com.hotpotato.core.mongo.Collection$Cursor.toList(Collection.scala:205)

.....

Any suggestions on what I'm doing wrong?

Thanks,

Lincoln

[Eliot Horowitz]

It's possible it's broken with embedded fields. Could you try putting
gps at the top level? I'll test more tonight

On Mar 2, 2010, at 6:58 PM, Lincoln Hochberg <lin...@hotpotato.com>
wrote:

> --
> You received this message because you are subscribed to the Google
> Groups "mongodb-user" group.
> To post to this group, send email to mongod...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-user...@googlegroups.com
> .
> For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en
> .

[Eliot Horowitz]

That was the problem.

Fixed in master: http://jira.mongodb.org/browse/SERVER-694

On Tue, Mar 2, 2010 at 7:38 PM, Eliot Horowitz <elioth...@gmail.com> wrote:

It's possible it's broken with embedded fields. Could you try putting gps at the top level?  I'll test more tonight

On Mar 2, 2010, at 6:58 PM, Lincoln Hochberg <lin...@hotpotato.com> wrote:

Hey guys,

Thanks very much for 1.3.3 - it's awesome!

I'm having a little trouble with geospatial search, which I'm testing.

I created the following index:

{
 "name" : "location.gps_",
 "ns" : "test.events",
 "key" : {
   "location.gps" : "2d"
 }
}

However, when I try to use it with a query like this:

{ "location.gps" : { "$near" : { "lat" : 40.719745 , "lon" : -73.955041}} , "_id" : { "$nin" : [ "4b8acc6e35b304d9dbdf6d90" , "4b8c244d468e04d925c6b191" ]} , "state" : "pre"}

I get the following error from the java driver:

java.lang.RuntimeException: db error [can't find index starting point]
   at com.mongodb.DBApiLayer$MyCollection.find(DBApiLayer.java:321)
   at com.mongodb.DBCursor._check(DBCursor.java:253)
   at com.mongodb.DBCursor._hasNext(DBCursor.java:374)
   at com.mongodb.DBCursor._fill(DBCursor.java:442)
   at com.mongodb.DBCursor.toArray(DBCursor.java:472)
   at com.mongodb.DBCursor.toArray(DBCursor.java:461)

   at com.hotpotato.core.mongo.Collection$Cursor.toList(Collection.scala:205)

[Lincoln]

Thanks Eliot! 

[Lincoln]

Hey guys, I'm on 1.3.4 now but it seems like I still can't create the 2d index on an embedded field with ensureIndex.  Is this still a problem?

Thanks,

Lincoln

[Eliot Horowitz]

Shouldn't be, can you send the command you're using?

[Lincoln]

Hmmm, it seems to have something to do with the existing indexes on the collection.

If I do it on a new collection it works fine for embedded fields.  However, I have a collection with a bunch of existing indexes, some of which touch the same fields and something is going wrong.

I'm seeing this output after I call ensureIndex:

i.more()

> db.events.getIndexes()               

[

{

"name" : "_id_",

"ns" : "hotpotato_integration.events",

"key" : {

"_id" : 1

}

},

{

"name" : "visibility_1_editorial_1_state_1",

"ns" : "hotpotato_integration.events",

"key" : {

"visibility" : 1,

"editorial" : 1,

"state" : 1

}

},

{

"name" : "_id_1_state_1",

"ns" : "hotpotato_integration.events",

"key" : {

"_id" : 1,

"state" : 1

}

},

{

"name" : "hp_tags_-1_state_1_visibility_1_starts_at_1",

"ns" : "hotpotato_integration.events",

"key" : {

"hp_tags" : -1,

"state" : 1,

"visibility" : 1,

"starts_at" : 1

}

},

{

"name" : "_id_1_visibility_1",

"ns" : "hotpotato_integration.events",

"key" : {

"_id" : 1,

"visibility" : 1

}

},

{

"name" : "state_1_starts_at_1_activity.last_msg_at_1",

"ns" : "hotpotato_integration.events",

"key" : {

"state" : 1,

"starts_at" : 1,

"activity.last_msg_at" : 1

}

},

{

"name" : "state_1_starts_at_1_reminder_id_1",

"ns" : "hotpotato_integration.events",

"key" : {

"state" : 1,

"starts_at" : 1,

"reminder_id" : 1

}

},

{

"name" : "state_1_activity.last_msg_at_1",

"ns" : "hotpotato_integration.events",

"key" : {

"state" : 1,

"activity.last_msg_at" : 1

}

},

{

"name" : "reminder_id_1",

"ns" : "hotpotato_integration.events",

"key" : {

"reminder_id" : 1

}

},

{

"name" : "editorial_-1_state_1__id_1",

"ns" : "hotpotato_integration.events",

"key" : {

"editorial" : -1,

"state" : 1,

"_id" : 1

}

},

{

"name" : "_id_1_activity.participants_-1_activity.stats.total_-1_state_1",

"ns" : "hotpotato_integration.events",

"key" : {

"_id" : 1,

"activity.participants" : -1,

"activity.stats.total" : -1,

"state" : 1

}

},

{

"name" : "location.gps.lat_1_location.gps.lon_1_state_1_starts_at_-1__id_1",

"ns" : "hotpotato_integration.events",

"key" : {

"location.gps.lat" : 1,

"location.gps.lon" : 1,

"state" : 1,

"starts_at" : -1,

"_id" : 1

}

},

{

"name" : "location.gps.lat_1_location.gps.lon_1_state_1_starts_at_-1_visibility_1",

"ns" : "hotpotato_integration.events",

"key" : {

"location.gps.lat" : 1,

"location.gps.lon" : 1,

"state" : 1,

"starts_at" : -1,

"visibility" : 1

}

},

{

"name" : "_id_1_activity.first_msg_at_1",

"ns" : "hotpotato_integration.events",

"key" : {

"_id" : 1,

"activity.first_msg_at" : 1

}

},

{

"name" : "location.gps.lat_1_location.gps.lon_1_state_1_visibility_1_starts_at_-1",

"ns" : "hotpotato_integration.events",

"key" : {

"location.gps.lat" : 1,

"location.gps.lon" : 1,

"state" : 1,

"visibility" : 1,

"starts_at" : -1

}

}

]

> db.events.getIndexes().length

15

> 

> db.events.ensureIndex({"location.gps":"2d"})

i.more()

> 

> db.events.getIndexes().length               

15

Thanks,

Lincoln

[Eliot Horowitz]

Is there anything corresponding in the server log?

[Lincoln]

I should have thought to look there:

Thu Mar 18 18:43:52 building new index on { location.gps: "2d" } for hotpotato_integration.events

Thu Mar 18 18:43:52 Buildindex hotpotato_integration.events idxNo:15 { _id: ObjId(4ba274685d3d5be96860ca0e), ns: "hotpotato_integration.events", key: { location.gps: "2d" }, name: "location.gps_" }

Thu Mar 18 18:43:52  hotpotato_integration.system.indexes Assertion failure i.more() db/index_geo2d.cpp 400

0x4f5826 0x4fe334 0x65a14e 0x65b434 0x64d820 0x605567 0x6067e8 0x6077a3 0x609476 0x5f092c 0x5f55c0 0x69e8b2 0x6b7220 0x7fad8cdb0a04 0x7fad8c36f80d 

 /opt/hp/mongo/bin/mongod(_ZN5mongo12sayDbContextEPKc+0xe6) [0x4f5826]

 /opt/hp/mongo/bin/mongod(_ZN5mongo8assertedEPKcS1_j+0x154) [0x4fe334]

 /opt/hp/mongo/bin/mongod(_ZNK5mongo9Geo2dType5_hashERKNS_7BSONObjE+0x4e) [0x65a14e]

 /opt/hp/mongo/bin/mongod(_ZNK5mongo9Geo2dType7getKeysERKNS_7BSONObjERSt3setIS1_NS_22BSONObjCmpDefaultOrderESaIS1_EE+0xd4) [0x65b434]

 /opt/hp/mongo/bin/mongod(_ZNK5mongo9IndexSpec7getKeysERKNS_7BSONObjERSt3setIS1_NS_22BSONObjCmpDefaultOrderESaIS1_EE+0x40) [0x64d820]

 /opt/hp/mongo/bin/mongod(_ZN5mongo14fastBuildIndexEPKcPNS_16NamespaceDetailsERNS_12IndexDetailsEi+0x4e7) [0x605567]

 /opt/hp/mongo/bin/mongod [0x6067e8]

 /opt/hp/mongo/bin/mongod(_ZN5mongo11DataFileMgr6insertEPKcPKvibRKNS_11BSONElementEb+0x853) [0x6077a3]

 /opt/hp/mongo/bin/mongod(_ZN5mongo11DataFileMgr6insertEPKcRNS_7BSONObjEb+0x66) [0x609476]

 /opt/hp/mongo/bin/mongod(_ZN5mongo14receivedInsertERNS_7MessageERNS_5CurOpE+0x26c) [0x5f092c]

 /opt/hp/mongo/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERK11sockaddr_in+0x15b0) [0x5f55c0]

 /opt/hp/mongo/bin/mongod(_ZN5mongo10connThreadEv+0x232) [0x69e8b2]

 /opt/hp/mongo/bin/mongod(thread_proxy+0x80) [0x6b7220]

 /lib/libpthread.so.0 [0x7fad8cdb0a04]

 /lib/libc.so.6(clone+0x6d) [0x7fad8c36f80d]

Thu Mar 18 18:43:52 User Exception 10086:ns not found: hotpotato_integration.events.$location.gps_

Thu Mar 18 18:43:52    Caught Assertion in insert , continuing

Thu Mar 18 18:43:52 insert hotpotato_integration.system.indexes exception assertion db/index_geo2d.cpp:400 1ms

It seems to be saying it can't find the namespace, but if I simply change "2d" to 1 the ensureIndex call works fine.

Thu Mar 18 18:46:32 building new index on { location.gps: 1.0 } for hotpotato_integration.events

Thu Mar 18 18:46:32 Buildindex hotpotato_integration.events idxNo:15 { _id: ObjId(4ba275085d3d5be96860ca0f), ns: "hotpotato_integration.events", key: { location.gps: 1.0 }, name: "location.gps_1" }

Thanks,

Lincoln

[Eliot Horowitz]

That assert means that there is a bad object in the location field.
looks like an empty array or object.
i'll make the error message nicer.

[Lincoln]

So I need to find and clean up that data before I can create the index?

[Lincoln]

Actually I won't always have a location, and sometimes I have an empty object there.  Do I need to ensure that the field is not written at all if empty?

[Eliot Horowitz]

You can have null, but not an empty object.
I think i'll change it to make an empty object the same as null as
that seems a little strict.

[Eliot Horowitz]

Ok - fix is in master for empty geo fields.

On Thu, Mar 18, 2010 at 2:52 PM, Lincoln <linxb...@gmail.com> wrote:

[Lincoln]

That would be awesome, thanks Eliot.

Lincoln