Anyway to have mongos start before the config servers are up and running?

[wulfgar.pro]

I have three config servers running on VM's managed by Puppet. I like the idea of NO inter-dependencies amongst the VM's and their boot order; i.e. If the config servers are down, I want to start the server hosting mongos even though it won't be able to successfully connect. Is there some way of achieving this? Why doesn't mongos poll to establish a connection with the configuration servers?

[Adam C]

There is a somewhat related feature request here:

https://jira.mongodb.org/browse/SERVER-2613

Since the mongos requires the metadata to exist somewhere in order to start, that request focuses on providing an alternative and suggests using a mongos to restore a lost config server in the case of disaster recovery etc.

Instead of that, I think you are looking for a way to simply successfully start the mongos service and have it periodically poll for its config servers until they appear - correct?

Thinking about it, I suppose an active rejection of incoming connections, with an informative error and the ability to catch said error in the drivers would be a decent alternative.  That way, the client waits until the mongos is ready and can see its config servers before allowing queries etc.

Have I got your idea right?

Adam.

[Glenn Maynard]

On Mon, May 14, 2012 at 8:44 AM, Adam C <ad...@10gen.com> wrote:

Instead of that, I think you are looking for a way to simply successfully start the mongos service and have it periodically poll for its config servers until they appear - correct?

Thinking about it, I suppose an active rejection of incoming connections, with an informative error and the ability to catch said error in the drivers would be a decent alternative.  That way, the client waits until the mongos is ready and can see its config servers before allowing queries etc.

It could just refuse connections while it's waiting for the config server, which is equivalent (from the client's perspective) to what happens if mongos isn't running at all, except you don't need wrapper a script to keep restarting mongos while you wait for a config server to come online.  It does seem like mongos should deal with handling initial retries itself; it's easier for it to do it correctly.

I can see the use for returning errors, too, though--it's easier to tell what's wrong from the client side.

-- 
Glenn Maynard

[Adam C]

My argument in favor of errors rather than refusing connections would be to allow the driver to back off, handle a mongos-in-waiting differently than a mongos-down.  In any case, going to talk this one over internally and potentially add it as a feature request.  Will post back here shortly one way or the other.

Adam.

[Glenn Maynard]

On Mon, May 14, 2012 at 6:41 PM, Adam C <ad...@10gen.com> wrote:

My argument in favor of errors rather than refusing connections would be to allow the driver to back off, handle a mongos-in-waiting differently than a mongos-down.  In any case, going to talk this one over internally and potentially add it as a feature request.  Will post back here shortly one way or the other.

It also means mongos can retry when it's appropriate (config down), but not when it's inappropriate (fatal configuration error).  You can do that with scripts, with careful use of process exit values, but users really shouldn't have to worry about that.

--
Glenn Maynard

[wulfgar.pro]

I could open the feature request if need be.

[Adam C]

Actually, thinking about the graceful error type scenario, it would be a bad idea unless you are using safe writes (not the default).  A connection reset (socket exception) would still work, and in some cases be a quicker failure, depending on your OS and networking set up (think versus a black hole type scenario where the OS does not actively reject inbound connections).

[Glenn Maynard]

On Mon, May 14, 2012 at 7:07 PM, Adam C <ad...@10gen.com> wrote:

Actually, thinking about the graceful error type scenario, it would be a bad idea unless you are using safe writes (not the default).  A connection reset (socket exception) would still work, and in some cases be a quicker failure, depending on your OS and networking set up (think versus a black hole type scenario where the OS does not actively reject inbound connections).

I don't have a strong opinion of keeping the port closed (refuse connections until the config connection succeeds) vs. responding with an error, but either way, if you're running a server in an environment like the above you're pretty doomed no matter what you do.

Either way, having mongos handle config retries is better than making every user set up a wrapper script to do it, though.

--
Glenn Maynard

[Sosh]

Has there been any progress on this? Is it still necessaries to start up the config servers before the mongos servers?

[Adam C]

Yes it is still required to have at least one config server must be up before starting the mongos processes.

Adam