Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Safemode detected an error 'not master'. when primary switches from one instance to another
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  11 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Demitri Baroutsos  
View profile  
 More options Jul 3 2012, 6:36 am
From: Demitri Baroutsos <demitri.barout...@gmail.com>
Date: Tue, 3 Jul 2012 03:36:58 -0700 (PDT)
Local: Tues, Jul 3 2012 6:36 am
Subject: Safemode detected an error 'not master'. when primary switches from one instance to another

We're currently running all our MongoDB instances with version 2.0.5 of the
MongoDB server with replica-sets.
Each server is hosted on a separate machine to cater for fail-over.

We are using version 1.3.0.4309 of the C# Driver and we're seeing the
following error. It happens just after a network failure where one of the
server instances takes over as primary in the replica-set:

Safemode detected an error 'not master'. (Response was { "err" : "not
master", "code" : 10058, "n" : 0, "lastOp" : NumberLong(0), "connectionId"
: 140567, "ok" : 1.0 }).

Here's the stack-trace of that error occurrence:
   at
MongoDB.Driver.Internal.MongoConnection.SendMessage(MongoRequestMessage
message, SafeMode safeMode)
   at MongoDB.Driver.MongoCollection.InsertBatch(Type nominalType,
IEnumerable documents, MongoInsertOptions options)
   at MongoDB.Driver.MongoCollection.Insert(Type nominalType, Object
document, MongoInsertOptions options)
   at MongoDB.Driver.MongoCollection.Insert(Type nominalType, Object
document, SafeMode safeMode)
   at MongoDB.Driver.MongoCollection.Insert(Type nominalType, Object
document)
   at MongoDB.Driver.MongoCollection.Insert[TNominalType](TNominalType
document)

Our connection string has all three server names and ports as per the
documentation.

It appears as if the MongoDB Connection in the C# driver is not checking
that the one it knows to be primary is still the primary when doing a write
operation. The only way we can recover from this is to restart the
application pool the web application is running in, thereby forcing the
next request to re-create the database connection object and then it knows
which of the three server instance is now the primary again.

Is this perhaps a known issue with the version of the server software and
C# driver we are using? We are planning on updating the code-base to use
the latest version of the C# driver shortly.

Thanks in advance

Regards,
Demitri


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Scott Hernandez  
View profile  
 More options Jul 3 2012, 7:15 am
From: Scott Hernandez <scotthernan...@gmail.com>
Date: Tue, 3 Jul 2012 12:15:44 +0100
Local: Tues, Jul 3 2012 7:15 am
Subject: Re: [mongodb-csharp] Safemode detected an error 'not master'. when primary switches from one instance to another

How are you connecting to the server? What connection string (URI) or
constructor (MongoServerSettings) are you using?

On Tue, Jul 3, 2012 at 11:36 AM, Demitri Baroutsos <


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Demitri Baroutsos  
View profile  
 More options Jul 3 2012, 8:01 am
From: Demitri Baroutsos <demitri.barout...@gmail.com>
Date: Tue, 3 Jul 2012 05:01:34 -0700 (PDT)
Local: Tues, Jul 3 2012 8:01 am
Subject: Re: [mongodb-csharp] Safemode detected an error 'not master'. when primary switches from one instance to another

Our connection string format is as follows:

mongodb://mongoDBUser:MongoDBPassw...@10.40.123.4:27017,10.80.123.5:27017,10.80.123.6:27017/?safe=true;w=2;wtime outMS=2000;slaveOk=true

We keep a static instance of the server and database objects as follows:

        private static MongoServer _server;
        private static MongoServer DBServer
        {
            get
            {
                if (_server == null)
                {
                    _server =
MongoServer.Create(ConfigurationManager.ConnectionStrings["MyMongoConnectio nString"].ToString());
                }
                return _server;
            }
        }

        private static MongoDatabase _db;
        internal static MongoDatabase DB
        {
            get
            {
                if (_db == null)
                {
                    _db = DBServer.GetDatabase("MyMongoDBName");
                }
                return _db;
            }
        }

We fetch collections using the DB object as follows using generic types:

internal static MongoCollection GetCollection<T>()
{
    return DB.GetCollection(typeof(T), typeof(T).Name);

}

Could it be that the issue is caused by us keeping static instances of the
DB and Server objects in a web environment?

Just to re-iterate the issue around server instance connectivity. The web
server maintains connection with the current primary DB server. However the
two other instances (according to their logs) lose connectivity to the
current primary and one of them is elected as primary. Now when the web
server tries to write to the original primary it appears that it is still
seeing this instance as the primary instance and thus returns the Safemode
detected an error 'not master'. error when in fact, one of the other
servers has taken over as primary. Only when either re-starting the
app-pool on the web server or restarting the MongoDB instance that WAS the
primary resolves the exception error.

Regards,
Demitri


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
craiggwilson  
View profile  
 More options Jul 3 2012, 8:21 am
From: craiggwilson <craiggwil...@gmail.com>
Date: Tue, 3 Jul 2012 05:21:33 -0700 (PDT)
Local: Tues, Jul 3 2012 8:21 am
Subject: Re: [mongodb-csharp] Safemode detected an error 'not master'. when primary switches from one instance to another

I'm not sure if it is known issue or not with that  version, but I do know
a lot of work has gone into replica set connections since the release of
that version, which was a long time ago.  If you are planning on updating,
then that should fix the issue.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Robert Stam  
View profile  
 More options Jul 3 2012, 8:50 am
From: Robert Stam <rob...@10gen.com>
Date: Tue, 3 Jul 2012 08:50:04 -0400
Local: Tues, Jul 3 2012 8:50 am
Subject: Re: [mongodb-csharp] Safemode detected an error 'not master'. when primary switches from one instance to another

Sounds like you are expecting the driver to handle the failover from one
primary to another without throwing any exceptions at all, which is not the
case for several reasons:

1. checking whether the member we think is primary is still primary before
each operation would double the number of round trips to the server
2. failover is not instantaneous, until the election finishes there is no
primary

The driver does not currently retry any operations, so you will see some
exceptions in your application while the failover is in progress and your
application should be prepared to handle these. We think the application
should handle all exceptions because only the application knows:

1. whether it even wants to retry
2. what kinds of exceptions should be retried
3. how many times to retry
4. for how long to retry

Keep in mind that you could also be getting exceptions of other kinds now
and then (network down, etc...) so you need to be prepared to handle
exceptions anyway, and the exceptions that occur during failover from one
primary to another are just one kind of exceptions you need to handle.

You should upgrade to a current version of the driver soon anyway, but in
this case upgrading to 1.4.2 (or 1.5) won't change the behavior you are
describing, which is working as designed.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Demitri Baroutsos  
View profile  
 More options Jul 3 2012, 12:04 pm
From: Demitri Baroutsos <demitri.barout...@gmail.com>
Date: Tue, 3 Jul 2012 09:04:36 -0700 (PDT)
Local: Tues, Jul 3 2012 12:04 pm
Subject: Re: Safemode detected an error 'not master'. when primary switches from one instance to another

Thanks for the reply Robert.

We do cater for exceptions and don't expect the driver to do all the work
for us. However every subsequent request to the database fails when
attempting to write until we restart the app pool/IIS or the MongoDB
instance that used to be primary for that replica-set.

Regards,
Demitri


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Robert Stam  
View profile  
 More options Jul 3 2012, 12:21 pm
From: Robert Stam <rob...@10gen.com>
Date: Tue, 3 Jul 2012 12:21:07 -0400
Local: Tues, Jul 3 2012 12:21 pm
Subject: Re: [mongodb-csharp] Re: Safemode detected an error 'not master'. when primary switches from one instance to another

Can you try with a current version of the C# driver? Either 1.4.2 or 1.5.

I can't find any existing JIRA ticket that would be similar to what you are
describing.

On Tue, Jul 3, 2012 at 12:04 PM, Demitri Baroutsos <


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Demitri Baroutsos  
View profile  
 More options Jul 4 2012, 3:18 am
From: Demitri Baroutsos <demitri.barout...@gmail.com>
Date: Wed, 4 Jul 2012 00:18:10 -0700 (PDT)
Local: Wed, Jul 4 2012 3:18 am
Subject: Re: [mongodb-csharp] Re: Safemode detected an error 'not master'. when primary switches from one instance to another

Thanks Robert - we will try with the latest driver and see if the problem
persists.

Regards,
Demitri


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Demitri Baroutsos  
View profile  
 More options Jul 10 2012, 4:31 am
From: Demitri Baroutsos <demitri.barout...@gmail.com>
Date: Tue, 10 Jul 2012 01:31:35 -0700 (PDT)
Local: Tues, Jul 10 2012 4:31 am
Subject: Re: [mongodb-csharp] Re: Safemode detected an error 'not master'. when primary switches from one instance to another

Just an update - we needed a fix quite urgently so we deployed latest build
of our code (removed use of static instances of Database and Server
contexts, we create instances on-demand now) using the latest C# driver as
well.

So far we've not encountered any issues with fail-over causing problems.

Regards,
Demitri


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Robert Stam  
View profile  
 More options Jul 10 2012, 9:48 am
From: Robert Stam <rob...@10gen.com>
Date: Tue, 10 Jul 2012 09:48:59 -0400
Local: Tues, Jul 10 2012 9:48 am
Subject: Re: [mongodb-csharp] Re: Safemode detected an error 'not master'. when primary switches from one instance to another

Thanks for the update.

MongoServer.Create always returns the same instance of MongoServer when
called with the same parameters, so even though you may be calling
MongoServer.Create on demand, you're still reusing the single instance of
MongoServer.

On Tue, Jul 10, 2012 at 4:31 AM, Demitri Baroutsos <


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Demitri Baroutsos  
View profile   Translate to Translated (View Original)
 More options Jul 10 2012, 9:58 am
From: Demitri Baroutsos <demitri.barout...@gmail.com>
Date: Tue, 10 Jul 2012 15:58:19 +0200
Local: Tues, Jul 10 2012 9:58 am
Subject: Re: [mongodb-csharp] Re: Safemode detected an error 'not master'. when primary switches from one instance to another

Thanks for the confirmation and prompt responses Robert.

Demitri

On 10 July 2012 15:48, Robert Stam <rob...@10gen.com> wrote:

--
http://about.me/deems

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »