S3 reads are getting slow when I set request_timeout field

[Mayank Gupta]

Hi,

I am exploring S3 object storage backed Merge Tree tables and storage tiering using TTL move, where the first 5 days of data is stored on CH node local disks, and data older than 5 days is moved to Minio object storage (S3) using TTL moves. During the movement of data, I was seeing request timeout errors in Clickhouse and the transfer speed was very low. In order to fix this, I added the request_timeout property in s3 to 1 min. This fixed my slow writes problem.
Now, when I tried to read data using Select queries, the queries speed were very slow. I noticed that the value of request_timeout is directly proportional to the read time.

I am not sure if this is the desired behavior. Can someone confirm this? Also, how can we fix this issue?