question about downsampling

[Huafeng Lyu]

HI guys,

If I have an opentsdb instance running on host A, and its hbase storage is on host B, and I needs to do many downsampling queries. I noticed that the CPU usage on host B increased a lot, but not on host A. I'm curious - which part of computing used most CPU, (1) reading data from hbase, or (2) doing the aggregation/downsampling calculation?

Thanks.
--huafeng

[ManOLamancha]

On Monday, October 24, 2016 at 6:22:49 AM UTC-7, Huafeng Lyu wrote:

If I have an opentsdb instance running on host A, and its hbase storage is on host B, and I needs to do many downsampling queries. I noticed that the CPU usage on host B increased a lot, but not on host A. I'm curious - which part of computing used most CPU, (1) reading data from hbase, or (2) doing the aggregation/downsampling calculation?

Does the HBase CPU increase only during query times? HBase will usually exhibit high CPU when queries:

1) Include a lot of filters

2) Miss the memstore and block cache having to read from disk. (common in historical queries or if the write traffic is so high that it flushes recent data from the cache)

You may want to investigate your block cache hit ratios in HBase and try tuning them.