Mongo Backup suddenly stopped working - get Error starting fetches. RsId: rs0 - Err: no reachable servers

[jdamnation]

Hi group,

I have successfully set up a basic MongoDB, I have configured the monitoring agent and added the host to MMS. All this part was fine.

I then turned my attention to backup. I followed the helpful wizard guidelines, installed the backup agent, stuck card details in, and all appeared to work, with my replica syncing & backing up.

All of a sudden however, after a few hours of syncing, I started getting errors from MMS via email:

"Backup oplog is behind "

Then when I went to go check on the replication set status - gone. Nada.

Finally I went to go check the backup agent, MMS shows this running fine, with the last configuration call being very recent. 

The problem is evident in the backup agent logs - shown below.

When on the server I can ping / resolve api-backup.mongodb.com no problem at all.

/etc/mongodb-mms/backup-agent.config contains the line:

#

# Hostname of the MMS Backup web server.

#

mothership=api-backup.mongodb.com

So that seems fine.

So am really confused as to what's happened, and rather concerned that a backup would simply vanish like this.. I have done my due diligence, scanned FAQs, help, forums, but can not find any answers.

Cheers,

JD

Error: Error starting fetches. RsId: rs0

Error dialing to mongo. Err: Error dialing to replica set. RsId: rs0 Hosts: [] Err: no reachable servers
	at backup-agent/components/replica_set.go:39
	at backup-agent/components/replica_set.go:55
	at backup-agent/components/missing.go:41
	at 10gen.com/backup-agent/backup-agent.go:166
	at 10gen.com/backup-agent/backup-agent.go:113
	at 10gen.com/backup-agent/backup-agent.go:62
	at 10gen.com/backup-agent/backup-agent.go:386
	at pkg/runtime/proc.c:182
	at pkg/runtime/proc.c:1223
	at backup-agent/components/missing.go:43
	at 10gen.com/backup-agent/backup-agent.go:166
	at 10gen.com/backup-agent/backup-agent.go:113
	at 10gen.com/backup-agent/backup-agent.go:62
	at 10gen.com/backup-agent/backup-agent.go:386
	at pkg/runtime/proc.c:182
	at pkg/runtime/proc.c:1223

[Ralf Kistner]

Hi,

I'm getting a similar error - received the same email at 2014/02/07 10:48 UTC this morning (3 hours ago). There is no problem with MMS for the same replicaset, and the MMS agent is on the same node as the backup agent.

The error when the backup first failed:

[2014/02/07 05:08:56] [agent.oplog.rs0.error] [backup-agent/components/oplog.go:241] Tail error. Err: EOF

And now:

[2014/02/07 06:38:52] [agent.error] [10gen.com/backup-agent/backup-agent.go:120] Error starting tails. RsId: rs0

Error dialing to mongo. Err: Error dialing to replica set. RsId: rs0 Hosts: [<node1>:27017 <node2>:27017 <node3>:27017] Err: no reachable servers

Regards,

Ralf Kistner

[Meghan Gill]

Can you please file a ticket in the MMS support jira with the details?

https://jira.mongodb.org/browse/MMSSUPPORT

Thanks,

Meghan

--
--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb
 
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Andrew Ryder]

Hi JD, Ralf,

Were you able to raise a help ticket in MMS Support Jira as Meghan suggested?

The reason this is requested is because that is a private forum where we can obtain group and machine details. These will help us figure out what happened.

Kind regards,
Andrew

[Ralf Kistner]

Hi Andrew,

Cailin Nelson helped us to sort it out - in our case it was an issue with some hosts having SSL enabled, and some not (an unusual setup that we'll only have during a migration process).

Regards,

Ralf

--
--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb
 
---

You received this message because you are subscribed to a topic in the Google Groups "mongodb-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mongodb-user/B2_d8JHxxZI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mongodb-user...@googlegroups.com.

[Kumar]

Hi,

Getting same error after enable the SSL, Backup agent not working. getting the error  "Error starting tails. RsId: XXXXX_Set1".

Please suggest.

[Kevin Adistambha]

Hi Kumar

Please note that you’re replying to a thread that is 4 years old.

MongoDB MMS was also changed to Cloud Manager in 2015, so any information in this thread would be very outdated.

If you’re having issues with Cloud Manager, please create a support ticket in the Cloud interface.

Best regards
Kevin

​