oplog size

[davidmo]

we are currently on version 2.4.8. i have a replicate set consisting of a primary and 2 secondaries. there is a process which when it runs causes the oplog window to go from 75 hours to 1.5.
the lag is less than 1 minute. should i increase the size of the oplog so it never goes this low ? if the data being processed keeps growing then i have to keep monitoring the oplog window and increase it from time to time ? or do i just make it enormous now and take the space away from the user databases ?
i am not sure what to do here, any insight would be greatly appreciated.  thanks

[davidmo]

hi folks. any thoughts on this ? should i really dramatically increase the size of the oplog for an every few days batch process ? i say dramatically because it is 16Gb now and it is going down to 1.5 hrs so i am thinking i need to double the size.

thanks

[s.molinari]

I don't know the answers to your questions, but out of curiosity, could you explain a bit about the batch process happening causing the flood of updates or writes? What is happening or being done?

Scott

[Asya Kamsky]

Yes, 100% yes, you should increase the size of your oplog so that
during its most intensive writing periods the replication window never
goes this short.

Imagine that your secondary or network fails right at the beginning of
this intensive load. It comes back an hour and a half later and ...
you have to do a full resync because the secondary fell off the oplog!

Asya

> --
> You received this message because you are subscribed to the Google Groups
> "mongodb-user"
> group.
>
> For other MongoDB technical support options, see:
> http://www.mongodb.org/about/support/.
> ---
> You received this message because you are subscribed to the Google Groups
> "mongodb-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mongodb-user...@googlegroups.com.
> To post to this group, send email to mongod...@googlegroups.com.
> Visit this group at http://groups.google.com/group/mongodb-user.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mongodb-user/5c959fda-12fa-4872-9630-140bed493b2c%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[davidmo]

scott, thank you for answering. the query is generally an update with a set of millions of records. i do not have the exact query yet. i will get it and post it if i can.

asya, thank you for your response. i will increase it.

On Friday, December 12, 2014 8:45:40 AM UTC-5, davidmo wrote: