How to find the datapoints per sec which are written to hbase through Telnet and HTTP request?

[shanky goyal]

We tried using this expression to get an average of the number of data points avg(q("sum:rate:tsd.hbase.rpcs.batched","1h","")*1024). Does this give me the exact count?

The other thing i tried is using avg(q("sum:rate:tsd.rpc.received{type=put|telnet}","1h","")) this gives a different value than the above one.

avg(q("sum:rate:tsd.hbase.rpcs.batched","1h","")*1024) = 52.89k

avg(q("sum:rate:tsd.rpc.received","1h","")) = 10.67k

My understanding on tsd.rpc.received metric is that it gives the RPC request received by TSDB and tsd.hbase.rpcs.batched gives the batches send to hbase by the client. So i thought more or less these 2 metrics should give us the same count.

How are these two metrics different? And how different are the results suppose to be?

The other question i have is if one component is sending a certain number ( say 200 request per sec) of request to Opentsdb in 1 sec then will that be one connection per request or one connection per component which is sending the data?

[ManOLamancha]

On Monday, December 5, 2016 at 9:42:17 PM UTC-8, shanky goyal wrote:

We tried using this expression to get an average of the number of data points avg(q("sum:rate:tsd.hbase.rpcs.batched","1h","")*1024). Does this give me the exact count?

That will give you the number of calls sent to HBase. OpenTSDB batches the data points (by default) before sending them to HBase to reduce write load.

 

The other thing i tried is using avg(q("sum:rate:tsd.rpc.received{type=put|telnet}","1h","")) this gives a different value than the above one.

This measures the number of calls made to your TSD over the telnet interface to write data. It's probably closer to the number of individual data points but if use the HTTP interface for writing data (or the JAVA API), this will miss it.

 

avg(q("sum:rate:tsd.hbase.rpcs.batched","1h","")*1024) = 52.89k

avg(q("sum:rate:tsd.rpc.received","1h","")) = 10.67k

This would also include queries and other calls to the TSD.

My understanding on tsd.rpc.received metric is that it gives the RPC request received by TSDB and tsd.hbase.rpcs.batched gives the batches send to hbase by the client. So i thought more or less these 2 metrics should give us the same count.

Also, these are counters so you want to convert them to rates and I'm not sure how rates play into the Graphite style interface yet.

 

How are these two metrics different? And how different are the results suppose to be?

I think the rate issue is hitting you, try them as rates and you should be good.

And if you're looking for the number of individual writes, the best query would be "m=sum:rate:tsd.hbase.rpcs{type=put}". That will track the individual put requests, one per data point, sent to HBase. This should be close to 1024X the batched value for a really busy TSD. (Also change to "type=append" if you're using appends)

 

The other question i have is if one component is sending a certain number ( say 200 request per sec) of request to Opentsdb in 1 sec then will that be one connection per request or one connection per component which is sending the data?

That depends on the type of connection, Telnet or HTTP. The former should increment per put request and the latter would increment per HTTP call.