Create index on oplog

[Lianghong Xu]

Hi,

I understand that oplog is designed to be append-only and supports very efficient sequential read from the tail. However, I would like to implement a feature in mongod that needs to query oplog entries by the "_id" field. This would lead to a lot of random accesses to the oplog and thus is very inefficent.

It would be ideal if mongodb supports index on the oplog to solve my problem, but unfortunately, it deliberately disallows doing so for its own design considerations. I'm wondering if there is any walk-around to this issue? To be more specific, which part of the server code do I need to modify to enable indexes on the oplog?

Below is the result when I first tried to create an index on the oplog:

rs0:PRIMARY> db.oplog.rs.ensureIndex({"_id" : 1})
{ "ok" : 0, "errmsg" : "cannot create indexes on the oplog", "code" : 67 }

I then modified the code in db/catalog/index_catalog.cpp to comment out the following:

if ( nss.isOplog() )
    return Status( ErrorCodes::CannotCreateIndex,
        "cannot create indexes on the oplog" );

However, it still doesn't work (as expected though).

rs0:PRIMARY> db.oplog.rs.ensureIndex({"_id" : 1})
{
    "createdCollectionAutomatically" : false,
    "numIndexesBefore" : 0,
    "ok" : 0,
    "errmsg" : "E11000 duplicate key error index: local.oplog.rs.$_id_ dup key: { : null }",
    "code" : 11000
}

Any help would be very much appreciated!!

Thanks!

Lianghong

[Eric Milkie]

Hi Lianghong,

Oplog entries do not have an "_id" field.  Also, indexes on _id are automatically created as unique indexes.  This explains why you are getting that error, as more than one document in the collection has the same (null) value for "_id".   You will need to modify the oplog format in order to have an index on a field named "_id".

-Eric

--
You received this message because you are subscribed to the Google Groups "mongodb-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-dev...@googlegroups.com.
To post to this group, send email to mongo...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-dev.
For more options, visit https://groups.google.com/d/optout.

[Lianghong Xu]

Ah, that's a really stupid mistake. Thanks for point out that there is no "_id" field in the oplog :)

I intended to create indexes on the "o._id" field in the oplog. Below is the result:

rs0:PRIMARY> db.oplog.rs.ensureIndex({"o._id" : 1})
{
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 0,
"numIndexesAfter" : 1,
"ok" : 1
}

rs0:PRIMARY> db.system.indexes.find()
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "local.startup_log" }
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "local.system.replset" }
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "local.me" }
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "local.slaves" }
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "local.replset.minvalid" }
{ "v" : 1, "key" : { "o._id" : 1 }, "name" : "o._id_1", "ns" : "local.oplog.rs" }

Can I safely assume that the index has been successfully created on the "o._id" field? I'm asking this because I see from somewhere else that indexes created on the oplog would never get updated.

Thanks!

Lianghong

[Eric Milkie]

In the 2.4 series, it was definitely the case that any indexes created on the oplog would not be updated.  In 2.6, I believe that indexes may now be updated properly, if you manage to get them created; this has never been tested explicitly though.

Also, note that it is still possible in MongoDB to replicate objects without _id fields, although it is problematic for obvious reasons.

-Eric

[Lianghong Xu]

Hi Eric,

I was able to create the index, but it seems that it causes a fatal error in mongod every time I try to update or insert to the database. Any idea what is going on and where to start to fix the problem?

Thanks!

Lianghong

Below is the error message:

rs0:PRIMARY> db.oplog.rs.getIndexes()

[

{

"v" : 1,

"key" : {

"o._id" : 1

},

"name" : "o._id_1",

"ns" : "local.oplog.rs"

}

]

rs0:PRIMARY> 

rs0:PRIMARY> 

rs0:PRIMARY> use test

switched to db test

rs0:PRIMARY> db.testData.insert({"text" : "hello"})

2014-09-10T16:52:50.480-0400 DBClientCursor::init call() failed

2014-09-10T16:52:50.481-0400 Error: error doing query: failed at src/mongo/shell/collection.js:258

2014-09-10T16:52:50.483-0400 trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed

2014-09-10T16:52:50.484-0400 warning: Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused

2014-09-10T16:52:50.484-0400 reconnect 127.0.0.1:27017 (127.0.0.1) failed failed couldn't connect to server 127.0.0.1:27017 (127.0.0.1), connection attempt failed

2014-09-10T16:52:50.486-0400 trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed

2014-09-10T16:52:50.487-0400 warning: Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused

2014-09-10T16:52:50.487-0400 reconnect 127.0.0.1:27017 (127.0.0.1) failed failed couldn't connect to server 127.0.0.1:27017 (127.0.0.1), connection attempt failed

> 

bye

The call stack is below:

2014-09-10T17:03:55.789-0400 [conn23] Assertion failure _indexCatalog.numIndexesTotal() == 0 src/mongo/db/catalog/collection.cpp 160

2014-09-10T17:03:55.802-0400 [conn23] 

 0xf06206 0xeb8baf 0xea1092 0x8cf104 0xc4c31c 0xc47111 0x980b7a 0x981110 0x982004 0x982939 0x9847f5 0x9974ed 0x998019 0x998701 0xbc5db8 0xa97d2a 0xa99723 0x7ff74a 0xecbbb7 0x7f2cca4860a4

----- BEGIN BACKTRACE -----

{"backtrace":[{"b":"400000","o":"B06206"},{"b":"400000","o":"AB8BAF"},{"b":"400000","o":"AA1092"},{"b":"400000","o":"4CF104"},{"b":"400000","o":"84C31C"},{"b":"400000","o":"847111"},{"b":"400000","o":"580B7A"},{"b":"400000","o":"581110"},{"b":"400000","o":"582004"},{"b":"400000","o":"582939"},{"b":"400000","o":"5847F5"},{"b":"400000","o":"5974ED"},{"b":"400000","o":"598019"},{"b":"400000","o":"598701"},{"b":"400000","o":"7C5DB8"},{"b":"400000","o":"697D2A"},{"b":"400000","o":"699723"},{"b":"400000","o":"3FF74A"},{"b":"400000","o":"ACBBB7"},{"b":"7F2CCA47E000","o":"80A4"}],"processInfo":{ "mongodbVersion" : "2.7.4-pre-", "gitVersion" : "23aa86b2f40897b013807b0d7c25417e7cfdaf12", "uname" : { "sysname" : "Linux", "release" : "3.1.0-1-amd64", "version" : "#1 SMP Tue Jan 10 05:01:58 UTC 2012", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "2EA619562CDBC48BA1CA7C073470ECA3BE8A5AA4" }, { "b" : "7FFF9E8FF000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "2F7A3B9E4817E62BEB9F13E58DB38AD67BC060B3" }, { "b" : "7F2CCA47E000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "08875C338A3B9CE4DDE1DCE3095373D64206A71D" }, { "b" : "7F2CCA276000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "CEDD7190EA3C29D73A1D4F63D620313614888AFD" }, { "b" : "7F2CCA072000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "067178338EB2BFE274949EA9E9E7E5349A1853EF" }, { "b" : "7F2CC9D67000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "5F040D102396D62ABA1156AFB53B6BEA1F8B65FB" }, { "b" : "7F2CC9A66000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "0B5F6404623D720E1D76AACF50CC86C64AFEF5BC" }, { "b" : "7F2CC9850000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "DD2004696A44E143226E8B9E980E08FE0091AC21" }, { "b" : "7F2CC94A7000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "B982C6061E89C13292F822B9FD603CC979B1D0D3" }, { "b" : "7F2CCA69B000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "7A88297C86EF41B060AA7D92FBDD48234591FAE0" } ] }}

 mongod(_ZN5mongo15printStackTraceERSo+0x26) [0xf06206]

 mongod(_ZN5mongo10logContextEPKc+0xCF) [0xeb8baf]

 mongod(_ZN5mongo12verifyFailedEPKcS1_j+0xC2) [0xea1092]

 mongod(+0x4CF104) [0x8cf104]

 mongod(+0x84C31C) [0xc4c31c]

 mongod(_ZN5mongo4repl5logOpEPNS_16OperationContextEPKcS4_RKNS_7BSONObjEPS5_Pbb+0x101) [0xc47111]

 mongod(_ZN5mongo18WriteBatchExecutor13execOneInsertEPNS0_16ExecInsertsStateEPPNS_16WriteErrorDetailE+0x49A) [0x980b7a]

 mongod(_ZN5mongo18WriteBatchExecutor11execInsertsERKNS_21BatchedCommandRequestEPSt6vectorIPNS_16WriteErrorDetailESaIS6_EE+0x330) [0x981110]

 mongod(_ZN5mongo18WriteBatchExecutor11bulkExecuteERKNS_21BatchedCommandRequestEPSt6vectorIPNS_19BatchedUpsertDetailESaIS6_EEPS4_IPNS_16WriteErrorDetailESaISB_EE+0x34) [0x982004]

 mongod(_ZN5mongo18WriteBatchExecutor12executeBatchERKNS_21BatchedCommandRequestEPNS_22BatchedCommandResponseE+0x639) [0x982939]

 mongod(_ZN5mongo8WriteCmd3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x285) [0x9847f5]

 mongod(_ZN5mongo12_execCommandEPNS_16OperationContextEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x3D) [0x9974ed]

 mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x8F9) [0x998019]

 mongod(_ZN5mongo12_runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x201) [0x998701]

 mongod(_ZN5mongo11newRunQueryEPNS_16OperationContextERNS_7MessageERNS_12QueryMessageERNS_5CurOpES3_+0x1128) [0xbc5db8]

 mongod(+0x697D2A) [0xa97d2a]

 mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xA63) [0xa99723]

 mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0xAA) [0x7ff74a]

 mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x4B7) [0xecbbb7]

 libpthread.so.0(+0x80A4) [0x7f2cca4860a4]

-----  END BACKTRACE  -----

2014-09-10T17:03:55.803-0400 [conn23] SEVERE: Fatal DBException in logOp(): 0 assertion src/mongo/db/catalog/collection.cpp:160

2014-09-10T17:03:55.811-0400 [conn23] SEVERE: terminate() called.

--
You received this message because you are subscribed to a topic in the Google Groups "mongodb-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-dev/OyBM3qmAoac/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongodb-dev...@googlegroups.com.

[Eric Milkie]

That is happening because the oplog writer is using a function to insert that doesn't support updating indexes.

See oplog.cpp:261, where it is calling the flavor of insertDocument() that takes an OplogDocWriter.  That particular flavor expects no indexes on the collection being inserted to, as you can see at collection.cpp:161 (the comment says we haven't implemented this functionality yet, because it hasn't been needed).  The other insertDocument() implementations in that file do support updating indexes.

Hope that helps!

-Eric