Data Recover using Delayed member

[Felipe Esteves]

Hi,

I'm trying to use a delayed replicaSet member for data-restore purposes, but unsuccessful so far.

In my scenario, I have:

a primary node

a delayed node (600 seconds)

and a arbiter node.

In the primary node, i create the "test" database,  then do some inserts in a test collection.

After 10 minutes, I can query the test database in the delayed mongo shell, as expected.

Then I drop the test database in the primary node, and it continues  to exist in the delayed, as expected.

What I want, at this point, is to turn the delayed in the new replicaSet primary, so I have the test database again.

I've tried two different approaches:

First, from the primary, I run a rs.reconfig raising the priority of the delayed member and setting slaveDelay=0. In this way, it becomes the primary, but "loses" the dropped database, going to the same state as the original  primary. What I expected, is to have  the delayed data as the new primary, i.e, the "test" database exists.

Second, I've ran the reconfig from the delayed member, using the force parameter in the rs.reconfig. The results were the same, delayed becomes primary, but as it happens, the "test" database also disappears.

Am I doing something wrong? Is it not possible to reconfig the replica as I'm willing to?

Mongo Version is 3.4.x, by the way.

cheers,

Felipe Esteves

Tecnologia

felipe....@b2wdigital.com

[Rhys Campbell]

I'm not entirely sure this siuation would a supported scenario.  I guess the mongod instance might apply the oplog in full when you promote it PRIMARY hence the missing 'test' database. You might have to drop and recreate the oplog. 

https://docs.mongodb.com/manual/core/capped-collections. 

MongoDB generally recommend not messing around with this. Make sure you do lots of testing if you go this way.

Would it not be easier to mongodump the test database and then recreate it on the existing primary? That way you'll avoid having to resync all the other members so it migth be quicker anyway.

Rhys

[Felipe Esteves]

Thanks for the reply, Rhys, I'm suspecting the oplog might be the cause indeed.

The problem for me to execute de dump/restore is the potential size of the database. In some scenarios, it may take too long to be able to resume operations, making it much easier if we can just turn the actual delayed into primary.

Felipe Esteves

Tecnologia

felipe....@b2wdigital.com

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/896b95c1-5c88-4254-ac41-bd0f2b202a6e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

 

---

Esta mensagem pode conter informações confidenciais e somente o indivíduo ou entidade a quem foi destinada pode utilizá-la. A transmissão incorreta da mensagem não acarreta a perda de sua confidencialidade. Caso esta mensagem tenha sido recebida por engano, solicitamos que o fato seja comunicado ao remetente e que a mensagem seja eliminada de seu sistema imediatamente. É vedado a qualquer pessoa que não seja o destinatário usar, revelar, distribuir ou copiar qualquer parte desta mensagem. Ambiente de comunicação sujeito a monitoramento.

This message may include confidential information and only the intended addresses have the right to use it as is, or any part of it. A wrong transmission does not break its confidentiality. If you've received it because of a mistake or erroneous transmission, please notify the sender and delete it from your system immediately. This communication environment is controlled and monitored.

B2W Digital

 

[Rhys Campbell]

This is from a while ago but it might still work...

https://groups.google.com/forum/#!topic/mongodb-user/w3sjxjSWJb0

You could edit the oplog entry containing the drop command. After promoting to PRIMARY you'd have to do a full resync to get the test database back on your primaries. In this case it may be quicker, and safer, to do a mongodump after all.

[Rhys Campbell]

"full resync to get the test database back on your primaries. " <- Obviously I meant SECONDARIES.

Cheers,

Rhys

[Wan Bachtiar]

What I want, at this point, is to turn the delayed in the new replicaSet primary, so I have the test database again.

Hi Felipe,

If you’re only interested in restoring the accidentally dropped test database, dumping the database from the delayed secondary and then restoring it to the primary would be preferable.

1. Stop the delayed secondary before the drop on test database has been applied.
2. Stand up the delayed secondary as a standalone
3. mongodump test database from the standalone.
4. mongorestore the backup data back into the replica set primary.
5. Add back the delayed secondary back into the replica set.

You also need to consider operations which may have been applied before the drop but have not been applied to the delayed secondary. Depending on the nature of these operations, you may utilise the oplog to restore them after the database has been restored.

Is it not possible to reconfig the replica as I’m willing to?

Note that if you forced the delayed secondary to become PRIMARY without having it to catch up first, you are rolling back the entire deployment to a point in time in the past. i.e. you may also lost other operations on other databases test2, test3, etc.

Also worth pointing out that afterward, the previous PRIMARY member would likely to perform initial-sync 

In my scenario, I have:

In addition, the minimum recommended configuration for a replica set is a three member replica set with three data-bearing members.

You may also be interested in reviewing various MongoDB backup methods.

Regards,
Wan.

​