keep printing warning msg if cassandra restart with IP changes

[Keith Lu]

hi, folks, 

i face an issue which below msg keeps printing due to cassandra IP changes.

it seems the datastax driver rescheduler to close the previous IP info.

just wonder whether this is the expected behavior?

or datastax should check new cassandra IP address?

[old IP]

Name: cassandra.ccmts-infra.svc.cluster.local

Address: 192.168.8.212

Name: cassandra.ccmts-infra.svc.cluster.local

Address: 192.168.6.220

Name: cassandra.ccmts-infra.svc.cluster.local

Address: 192.168.7.213

[new IP]

Name: cassandra.ccmts-infra.svc.cluster.local

Address: 192.168.6.251

Name: cassandra.ccmts-infra.svc.cluster.local

Address: 192.168.8.2

Name: cassandra.ccmts-infra.svc.cluster.local

Address: 192.168.7.9

1599157314.958 [WARN] (src/connection_pool.cpp:258:void cass::ConnectionPool::on_reconnect(cass::DelayedConnector*)): Connection pool was unable to reconnect to host 192.168.6.220 because of the following error: Connection timeout

1599157314.958 [WARN] (src/connection_pool.cpp:258:void cass::ConnectionPool::on_reconnect(cass::DelayedConnector*)): Connection pool was unable to reconnect to host 192.168.7.213 because of the following error: Connection timeout

1599157318.180 [WARN] (src/connection_pool.cpp:258:void cass::ConnectionPool::on_reconnect(cass::DelayedConnector*)): Connection pool was unable to reconnect to host 192.168.8.212 because of the following error: Connection timeout

[Keith Lu]

hi, folks, 

is this warning msg a must-have item? it is printed every 7 secs and not stopped. 

is this an issue or any API to clean up the old Cassandra reconnection behavior?

from cpp-driver/include/cassandra.h, only few API to adjust reconnection setup (new or old???).

it seems the old Cassandra IP should be clean up after a period of time.

below case is when only one Cassandra server is down with changing to new IP.

Name: cassandra.ccmts-infra.svc.cluster.local
Address: 192.168.6.155
Name: cassandra.ccmts-infra.svc.cluster.local
Address: 192.168.8.132
Name: cassandra.ccmts-infra.svc.cluster.local
Address: 192.168.7.142 -> 192.168.7.186

1599685662.701 [WARN] (src/cluster.cpp:422:void cass::Cluster::internal_notify_up(const cass::Address&, const Ptr&)): Attempting to mark host 192.168.7.186 that we don't have as UP
1599685666.298 [WARN] (src/connection_pool.cpp:258:void cass::ConnectionPool::on_reconnect(cass::DelayedConnector*)): Connection pool was unable to reconnect to host 192.168.7.142 because of the following error: Connection timeout

1599685883.373 [WARN] (src/connection_pool.cpp:258:void cass::ConnectionPool::on_reconnect(cass::DelayedConnector*)): Connection pool was unable to reconnect to host 192.168.7.142 because of the following error: Connection timeout
1599686870.723 [WARN] (src/connection_pool.cpp:258:void cass::ConnectionPool::on_reconnect(cass::DelayedConnector*)): Connection pool was unable to reconnect to host 192.168.7.142 because of the following error: Connection timeout

……………….never ends       ………………...... keep printing...…

[Michael Penick]

Has that node been removed from the cluster? If so a remove event should have stopped the reconnection process.

Have you tried using `cass_cluster_set_exponential_reconnect()`?

--
To unsubscribe from this group and stop receiving emails from it, send an email to cpp-driver-us...@lists.datastax.com.

--

Michael Penick

e. michael...@datastax.com
w. www.datastax.com

[Keith Lu]

hi, Michael,

thank you for reply.

my case is to restart Cassandra node ( still exist, but IP will be changed because k8s reassign new IP).

based on https://pkgs.org/download/cassandra-cpp-driver,

I use alpine 3.12, and seems driver side only support 2.10.
the API u mention is supported on 2.13.

 
I have tested on below API, it seems not workable.? the trace is dumpped every 7 secs on 2.10 version.
void

cass_cluster_set_reconnect_wait_time

( CassCluster * cluster, unsigned wait_time ) 
https://docs.datastax.com/en/developer/cpp-driver/2.10/api/struct.CassCluster/#function-cass_cluster_set_reconnect_wait_time

[Asif Iqbal K A]

Hi Michael,

    Even i see the same issue in our application as well; we are using driver version 2.12.

    Any way to clear that IP from driver?

[Asif Iqbal K A]

Hi All,

    Any one has any clue about this isssue?

[Asif Iqbal K A]

Hi Michael,

    Any clue about this issue.