ReadWriteBufferFromHTTP sending request to older IP

[Shiva Achari]

Hi,

We have cluster of 3 and table is distributed, One of the node Clickhouse server with ip 10.25.14.241 was down hence we had to do the following

we created a new instance and IP was changed

Updated config.xml and started the clickhouse server.

Please let me know where does ReadWriteBufferFromHTTP read the ip from or if we need to update the ip in zookeeper or anywhere else

Query is fine but detach partition is failing 

alter table table1 detach partition '2018-03-15'

ALTER TABLE table1

DETACH PARTITION '2018-03-15'

Received exception from server (version 1.1.54343):

Code: 209. DB::Exception: Received from localhost:9000, 127.0.0.1. DB::NetException. DB::NetException: Timeout: connect timed out: 10.25.14.241:9000: (10.25.14.241:9000).

0 rows in set. Elapsed: 10.046 sec.

Also in server logs i can see ReadWriteBufferFromHTTP trying to send request to an older IP of instance. 

2018.03.30 06:53:23.524414 [ 7 ] <Debug> DB.table1 (ReplicatedMergeTreeQueue): Not executing log entry for part 20180319_191_213_2 because another log entry for the same part is being processed. This shouldn't happen often.

2018.03.30 06:53:23.526906 [ 7 ] <Debug> DB.table1 (StorageReplicatedMergeTree): Fetching part 20180320_133_133_0 from /clickhouse/tables/shard1/table1/replicas/clickhouse2

2018.03.30 06:53:23.527787 [ 7 ] <Trace> ReadWriteBufferFromHTTP: Sending request to http://10.25.14.241:9009/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftables%2Fshard1%2Ftable1%2Freplicas%2Fclickhouse2&part=20180320_133_133_0&compress=false

2018.03.30 06:53:23.609272 [ 13 ] <Error> DB::StorageReplicatedMergeTree::queueTask()::<lambda(DB::StorageReplicatedMergeTree::LogEntryPtr&)>: Poco::Exception. Code: 1000, e.code() = 0, e.displayText() = Timeout: connect timed out: 10.25.14.241:9009, e.what() = Timeout

2018.03.30 06:53:23.660403 [ 8 ] <Error> DB::StorageReplicatedMergeTree::queueTask()::<lambda(DB::StorageReplicatedMergeTree::LogEntryPtr&)>: Poco::Exception. Code: 1000, e.code() = 0, e.displayText() = Timeout: connect timed out: 10.25.14.241:9009, e.what() = Timeout

2018.03.30 06:53:23.665590 [ 17 ] <Error> DB::StorageReplicatedMergeTree::queueTask()::<lambda(DB::StorageReplicatedMergeTree::LogEntryPtr&)>: Poco::Exception. Code: 1000, e.code() = 0, e.displayText() = Timeout: connect timed out: 10.25.14.241:9009, e.what() = Timeout

[Vitaliy Lyudvichenko]

You could run a system query SYSTEM DROP DNS CACHE which will drop ClickHouse internal DNS caches.

пятница, 30 марта 2018 г., 11:00:47 UTC+3 пользователь Shiva Achari написал: