Is there anyway we can query how many queries piling up in Druid?

827 views
Skip to first unread message

Noppanit Charassinvichai

unread,
Feb 5, 2016, 10:38:21 AM2/5/16
to Druid User
We have some long running queries and sometime that would slow down the cluster. We would like to start killing some of the slow running queries. I've just seen a way to cancel the query but is there anyway we can list how many queries running in Druid? http://druid.io/docs/0.8.3/querying/querying.html 

Thanks. You guys are awesome.  

charles.allen

unread,
Feb 5, 2016, 10:48:07 AM2/5/16
to Druid User
On historicals, you'll want to look at the segment/scan/pending metric http://druid.io/docs/latest/operations/metrics.html 

Noppanit Charassinvichai

unread,
Feb 5, 2016, 11:10:04 AM2/5/16
to Druid User
Thanks for your reply. I tried the query with our historical node and with GET http://historical-ip:8083/segment/scan/pending but I get 404 but I can go to http://historical-ip:8083/status and get result back. Not sure if my query is right?

charles.allen

unread,
Feb 5, 2016, 11:21:41 AM2/5/16
to Druid User
Please double-check the metrics docs:

Metrics are emitted as JSON objects to a runtime log file or over HTTP (to a service such as Apache Kafka). Metric emission is disabled by default.

The way you handle them is by setting up an emitter as per http://druid.io/docs/latest/configuration/#emitting-metrics

Noppanit Charassinvichai

unread,
Mar 2, 2016, 12:22:09 PM3/2/16
to Druid User
Hi,

Currently I have the emitter running already and it looks great. I was wondering if the log will be able to capture any pending query? Correctly me if I'm wrong, currently I'm seeing this 

{"feed":"metrics","timestamp":"2016-03-02T17:19:56.628Z","service":"druid/realtime","host":"address:8084","metric":"query/time","value":4,"context":"{\"finalize\":false,\"queryId\":\"1e2f29cf-b454-4e20-9a25-ede69cdc1217\",\"timeout\":30000}","dataSource":"firehose-web","duration":"PT3600S","hasFilters":"false","id":"1e2f29cf-b454-4e20-9a25-ede69cdc1217","interval":["2016-03-02T17:00:00.000Z/2016-03-02T18:00:00.000Z"],"remoteAddress":"address","type":"timeBoundary"}

Does this mean that it only shows the finished query? 

Fangjin Yang

unread,
Mar 9, 2016, 8:01:15 PM3/9/16
to Druid User
segment/scan/pending is closest to what you're looking for
Reply all
Reply to author
Forward
0 new messages