serverSelectionTimeout exception during failover

[Umesh Deshpande]

Hi,

I'm running a sharded cluster. In my python program, I connect to the mongos and insert documents. While it's writing documents in a loop, I kill the primary shard server. I expected the insert to throw the ServerSelectionTimeoutError exception, but it immediately throws an OperationFailure exception. How can I make the driver to wait for 5 seconds, until the serverSelectionTimeoutMS=5000 expires?

# To connect to the mongos
connection = pymongo.MongoClient("mongodb://localhost:27019", serverSelectionTimeoutMS=5000, connectTimeoutMS=500) 

Thanks,
Umesh

[Bernie Hackett]

Please provide the traceback from your application.

[Umesh Deshpande]

If I catch the pymongo.errors.OperationFailure exception and print the error. It shows the following message, where host1:27018 is the killed primary.

write results unavailable from host1:27018 :: caused by :: Location11002: socket exception [CONNECT_ERROR] for host1:27018

If I let it fail, without catching exception.

Unexpected error: <class 'pymongo.errors.WriteError'> , for index  349999

Traceback (most recent call last):

  File "loaddb.py", line 46, in <module>

    collection.insert(batch)      

  File "/usr/local/lib/python3.5/dist-packages/pymongo/collection.py", line 2469, in insert

    check_keys, manipulate, write_concern)

  File "/usr/local/lib/python3.5/dist-packages/pymongo/collection.py", line 611, in _insert

    _check_write_command_response(results)

  File "/usr/local/lib/python3.5/dist-packages/pymongo/helpers.py", line 315, in _check_write_command_response

    raise WriteError(error.get("errmsg"), error.get("code"), error)

pymongo.errors.WriteError: interrupted at shutdown

[Umesh Deshpande]

Here's the code snippet which threw the exception.

connection = pymongo.MongoClient("mongodb://localhost:27019", serverSelectionTimeoutMS=5000, connectTimeoutMS=500)

....

while True:

                try:

                    collection.insert(batch)

                    print('inserted doc batch')

                except pymongo.errors.ServerSelectionTimeoutError:

                    print('timeout error')

                    continue

                except pymongo.errors.OperationFailure as err:

                    print('operation failure: %s' % err)

                    continue

                break

...

[Bernie Hackett]

serverSelectionTimeoutMS configures how long PyMongo (or any other MongoDB driver) attempts to find a mongod (in the case of a replica set) or a mongos (in the case of a sharded cluster) that can accept the command your application is trying to execute.

An important thing to understand about serverSelectionTimeoutMS is that it applies only when selecting a server to send a command to. In your example server selection was successful, both in the driver and in mongos, but the operation failed on the shard because you shut down the shard server while it was executing the operation. MongoDB drivers do not retry failed operations for you. Neither does mongos. It is up to your application to decide when and how to retry failed operations like this.