Problem connecting to a partitioned local DC

[Alan Pound]

Hi

I've got two data centres in a NetworkTopologyStrategy POC with 1 replica in each for system_auth and 

other keyspaces.

I've a python app local to each DC that I'm connecting to the local replica in the manner below.

    cluster = Cluster( [ip_addr], auth_provider=PlainTextAuthProvider( username='cassandra', \

                password='cassandra'), \

                load_balancing_policy = DCAwareRoundRobinPolicy(local_dc='dc1' ), \

                default_retry_policy=DowngradingConsistencyRetryPolicy(), \

                protocol_version = 3)

    cdb = cluster.connect()

Once the cluster is connected, ordinary read and write operations survive partitioning of the two DCs, however if

the DCs cannot see each other at the initial connection, it throws an exception - but under the same 

circumstances cqlsh manages to startup and connect just fine.

It appears that the python driver is using a consistency requirement of QUORUM - is there any way of persuading it

to downgrade that requirement? (I know a replica set of 2 is a degenerate condition, but I'd really like to cope with

it).

Thanks

Alan

[Alan Pound]

PS. I should say it is Cassandra 2.1.2, python-driver '2.1.2.post', python 2.7.6 under ubuntu.

The error message when trying to connect immediately after the DCs have been partitioned is:

cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'10.202.124.253': OperationTimedOut('errors=Timed out creating new connection, last_host=None',)})

but after a little while it becomes:

cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'10.202.124.253': AuthenticationFailed(u'Failed to authenticate to 10.202.124.253: \ 

                code=0100 [Bad credentials] message="org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM"',)})

[Adam Holmberg]

It is best to increase the replication factor of system_auth when using PasswordAuthenticator. 

(I know a replica set of 2 is a degenerate condition, but I'd really like to cope with
it).

 

I'm not sure why you want to cope with it, but it might be of interest to you that PasswordAuthenticator requires a higher consistency level for the default 'cassandra' user, than others:

https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/auth/PasswordAuthenticator.java#L247-L253

Note that this is not the decision of the driver, but the authenticator implementation in Cassandra.

Regards,

Adam Holmberg

To unsubscribe from this group and stop receiving emails from it, send an email to python-driver-u...@lists.datastax.com.

[Alan Pound]

Wow.  Wickedly fast reply!

Yes, RF = number of replicas, I am tracking that.   I guess I've just got a non-typical application...

So I tried it again using a different set of credentials, and lo and behold - it works (who'd have guessed that one?)

Thanks very much, another little mystery solved ;^)

Regards

Alan