force opentsdb to write data to hbase (or disk) immediately

[Huafeng Lyu]

Hi,

Is there a method to force opentsdb to write datapoints it received to hbase (or, disk) immediately so that a query can return those new datapoints immediately? From what I saw, there's usually a delay of tens of seconds in my test environment.

Thanks for any tips.

--huafeng

[ManOLamancha]

On Tuesday, December 27, 2016 at 5:44:07 AM UTC-8, Huafeng Lyu wrote:

Is there a method to force opentsdb to write datapoints it received to hbase (or, disk) immediately so that a query can return those new datapoints immediately? From what I saw, there's usually a delay of tens of seconds in my test environment.

Hello, it sounds like you're seeing a GC or queue issue. In the TSD there is up to a 1 second delay by default as data is batched per region server to reduce the calls to HBase. If the TSD is GCin'g then you could see an increased delay. Enable Java GC logs and see if anything looks suspicious.

Also, HBase may be GCing as well so enable GC logging on the region servers as well and see if anything is happening there. And there is an RPC queue in HBase that you can monitor. If you see a large queue size or huge rpc processing latencies without GC, then you'll want to see if you have any disk or IO issues.