Cassandra tuning for Request timed out

[Hari Prasanth]

I am using Apache Cassandra as a data store for one of my project, It is a single node and running smoothly for 4 to 5 hours then I started getting the **Request timed out error**.

I observed the following in the last week and tuned all

1) GC pause was taking a long time and I tuned the GC pause and currently it is not even taking 2 seconds.   

PF the GC logs for reference : https://files.fm/u/h8b2mnwm#_

2) Memory is not even using above 5GB of 8GB allocated  
3) CPU with 8 cores seems fine.

If everything seems proper, What is the next thing I need to look in Cassandra to solve the **Request timed out.**     

**NOTE** :      
    i) READ / WRITE timeout is 60 seconds.     
    ii) Single Write with partition key gives Request timed out


Any help is appreciated.

Thanks,     
Harry

[Braulio Livio]

Hi,

You can try to increase the value of any parameter containing the "timeout" keyword in "../cassandra.yaml" file, such as "request_timeout_in_ms" or "write_request_timeout_in_ms".

This link below has the list of "cassandra.yaml" parameters

https://docs.datastax.com/en/cassandra/2.1/cassandra/configuration/configCassandra_yaml_r.html

I hope this help.

Regards,

Bráulio

--
You received this message because you are subscribed to the Google Groups "DataStax Java Driver for Apache Cassandra User Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-driver-user+unsubscribe@lists.datastax.com.

[Hari Prasanth]

Thanks But I already set 60 seconds to delay. Can you tell me what is Request timed out error in Cassandra and What is the remedy for this?

To unsubscribe from this group and stop receiving emails from it, send an email to java-driver-us...@lists.datastax.com.

[Braulio Livio]

Hi,

Try to Increase a little more the timeout parameters. 300 seconds or more. What type of operation are you performing? Inserting data? What size of your data? This error can be raised when inserting data with 1Mb or more.

Cassandra was projected to inserting e querying little chunks of data. Data with Mb or Gb of size are splited and inserted in different nodes in order to assure good performance.

When dealing with a single node, sometimes is necessary change the default values of timeout parameters by increasing them. 

Try to increase each timeout parameter in cassandra.yaml, one by one. It can works.

Braulio

[rgott...@sayeureqa.com]

Hi Hari,

You can resolve using below steps:

1. Java Driver connectivity:  Check below all options: in that main important for you is setConnectTimeoutMillis

Cluster cluster = Cluster.builder().addContactPoints(Seed IP#1,Seed IP#2)).

withPort(#port).withProtocolVersion(ProtocolVersion.V3)

.withLoadBalancingPolicy(new RoundRobinPolicy())

.withPoolingOptions(poolingOptions).withSocketOptions(

                new SocketOptions()

                        .setConnectTimeoutMillis(20000))

.build();

    Max using above option you are able to resolve. if still have the issue, use below option.

2. Alter table in CQL - Cassandra with below option:

alter table <table> with gc_grace_seconds = 0;

Let me know if you have still the issue.

Thanks

Ravi Gottipati

Ph: 9885858534

[Hari Prasanth]

How Connection time out and Request time out are same?

[Alexandre Dutra]

Hi,

ConnectTimeoutMillis and RequestTimeoutMillis are not the same. ConnectTimeoutMillis is how long the driver will wait for a new connection to be established before giving up; RequestTimeoutMillis is how long the driver will wait for a given Cassandra node to answer a query before giving up. What you need to tune here is RequestTimeoutMillis.

If your write request timeout server-side (write_request_timeout_in_ms) is 60 seconds, then you should set the driver's RequestTimeoutMillis to something slightly higher than that, maybe 65 seconds, so that a server-side timeout is not masked by a client-side timeout. 

If you still see timeouts after this, then it is likely that your are overloading the cluster and/or the driver, in which case you should throttle your write throughput.

Hope that helps,

Alexandre

--
You received this message because you are subscribed to the Google Groups "DataStax Java Driver for Apache Cassandra User Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-driver-us...@lists.datastax.com.

--

Alexandre Dutra

Driver & Tools Engineer @ DataStax