How to use mongorestore and mongodump to replay oplog from a source mongo server to a target mongo server (same way as mongooplog tool)?

[vaibhav singh]

Hi Guys,

As mongooplog tool is deprecated now. Is there any way to use mongodump and mongorestore in combination just to replay oplog from a source server to destination server . Which was possible

with earlier with mongooplog tool.   

[Dan Pasette]

Technically, you can do a `mongodump --oplog` on one mongodb server, remove all the collections from the dump directory except the oplog.bson file and then run `mongorestore --oplogReplay` pointing to a new server and it will replay just those oplog entries.

https://docs.mongodb.com/manual/reference/program/mongorestore/#cmdoption-oplogreplay

--
You received this message because you are subscribed to the Google Groups "mongodb-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-dev+unsubscribe@googlegroups.com.
To post to this group, send email to mongo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-dev/c5bf2e6d-b1e2-4bcd-a575-3334b561f51f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[vaibhav singh]

Its similar, but not the same as mongooplog which would do streaming replication. This functionality is useful to do migrations, PoC's, mirroring etc. Currently I have changed cursor in mongooplog tool to establish mirroring between two standalone mongo nodes . The code change made is mentioned below.

 

@@ -146,6 +172,6 @@ func buildTailingCursor(oplog *mgo.Collection,

        }

        // TODO: wait time

-       return oplog.Find(oplogQuery).Iter()

+       return oplog.Find(oplogQuery).Tail(-1)

 }

Could you add support for such streaming functionality in mongooplog and mongorestore?

On Saturday, 28 October 2017 03:43:43 UTC+5:30, Dan Pasette wrote:

Technically, you can do a `mongodump --oplog` on one mongodb server, remove all the collections from the dump directory except the oplog.bson file and then run `mongorestore --oplogReplay` pointing to a new server and it will replay just those oplog entries.

https://docs.mongodb.com/manual/reference/program/mongorestore/#cmdoption-oplogreplay

On Fri, Oct 27, 2017 at 9:28 AM, vaibhav singh <vaibhav....@gmail.com> wrote:

Hi Guys,

As mongooplog tool is deprecated now. Is there any way to use mongodump and mongorestore in combination just to replay oplog from a source server to destination server . Which was possible

with earlier with mongooplog tool.   

--
You received this message because you are subscribed to the Google Groups "mongodb-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-dev...@googlegroups.com.

[Dan Pasette]

Can i ask why you want mirroring functionality as opposed to regular replication?

To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-dev+unsubscribe@googlegroups.com.

To post to this group, send email to mongo...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-dev.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-dev/085e5224-ae2d-45e5-9166-ba1b4306b824%40googlegroups.com.

[david.b....@gmail.com]

Dan 

We talked about this before there are many good reasons someone would want such a tool for PITR (outside ops manager /atlas) where mongodump is only consistent backup with —oplog but you might need to use mongooplog to roll it forward to a point after the backup was taken before an issue. Or maybe they want cluster to cluster replication as part of a migration plan to have dual writes.

I can see many reasons this type of tool is important, given the trivial nature of the code I am  still confused on why MongoDB is so adamant to remove features from the community that other databases have (MySQLbinlog).

David Murphy  

Sent from my iPhone

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-dev/CACwpN6Pb_MFjzoHmPCChHa%3DfJ0sfrObQL_DfgEOjK5R-x2RVyQ%40mail.gmail.com.

[vaibhav singh]

Dan

We are currently using mongo 2.6 and planning to migrate to latest version of mongo. In order to do so we tested mongo version 3.0 with our production load by adding it to replica set as a secondary. But version higher than 3.0 couldn't be added to a 2.6 replica set thats why we created another cluster with mongo 3.2 with same databases as our 2.6 cluster and we are keeping this new cluster in sync with the old 2.6 cluster, using the mongooplog tool and then using flashback tool to send traffic from prod 2.6 cluster to this 3.2 new cluster to test whether it is able to handle the same amount of load or not .

Dan 

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-dev/085e5224-ae2d-45e5-9166-ba1b4306b824%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.