Re: Reliable way to get operation ids ("opid") of in-progress operations from Java driver in sharded mongo cluster

[Victor Voytovych]

Any chances that somebody answers this question?

On Monday, May 27, 2013 7:13:19 PM UTC+3, Victor Voytovych wrote:

Hi,

The goal is to periodically check mongodb for long running queries and be able to kill such queries (optional) using Java driver only.

The following function works great from mongodb shell:
function slowOps(threshold) {
    inprog = db.currentOp({'ns' : {$ne : ''}, 'secs_running' : { $gte : threshold}, 'op' : { $in : ['getmore', 'query']}}).inprog;
    var result = [];
    for (var i in inprog) {
        result.push(inprog[i].opid);
    }
    return result;
}

output:
mongos> slowOps(10)
[
        "m1-shard1set:1902811044",
        "m1-shard1set:1902811046",
        "m1-shard2set:-2101478616",
        "m1-shard2set:-2101478621",
        "m1-shard2set:-2101478620",
        "m1-shard3set:275802478"
]
and I'm able to kill operations using these ids.

But when I run the same function from Java driver using db.eval() function I'm getting numerical (not string) "opid"s and can't use them to kill running operations.

Could you please suggest me some reliable way to receive valid "opid"s of in-progress operations from Java driver in sharded mongo cluster.

Thank you in advance,
Victor

[Trisha Gee]

Hi Victor,

When I run this, whether it's from the shell or the Java driver, I get numeric opids, and I can kill the operations with these IDs.  Granted, I'm running a non-sharded system, and your results suggest that sharding might change the results.  Is your Java app connecting to the same system you got your original results from?

Any additional information in the form of code or error messages you get when trying to kill IDs would help enormously.

I feel I should also point out that killing operations arbitrarily from an application does seem like it might lead to unforeseen errors.  Is this approach really the only one you have to solve your problem?

Trisha

[Victor Voytovych]

In sharded mongodb cluster if you connect to mongos and run db.currentOp() commad from the mongo shell you'll get string opids like "m1-shard1set:1902811044".
In case if you connect directly to mongod and run the same command from the mongo shell you will be getting numerical (not string) opids like 1245845064.

When you use mongodb Java driver (2.11.1 in my case) and evaluate the same command using Java db.eval() function you will get numerical opids. In case if you will use received opids from JAva driver to run db.killOp() command using Java db.eval() function you will kill only one from three (because I have three shards) running operations. Another two will continue running and be visible from the mongodb shell.

Currently for me it looks like it's not possible to use currentOp() and killOp() commands effectively from Java driver for sharded mongodb cluster. I hope I'm wrong and you can suggest me some working solution.


Also, from another point of view I have sharded mongodb cluster with numerous dbs/collections. Some collections store hundreds of millions documents.
Multiple applications process/aggregate data and store them to mongodb. At the same time numerous clients access these data through REST API.

For multiple reasons for REST API I have to limit request duration (30-60 secs). Some queries require hours to complete and I need to cancel them when request timeout detected. Could you please suggest me the best reliable way to cancel query execution on mongodb side (for sharded cluster) from Java driver.

Currently, I have to periodically run db.killOp() from the mongodb shell (mongos) to kill my long running queries (secs_running > threshold).

Victor

[Trisha Gee]

I think it might be best to have a job running on the server to do this sort of thing, rather than remotely killing operations via the Java driver, the driver is really not designed to be an administration tool.

Alternatively, if you know what sort of queries take a long time (and I appreciate this is non-trivial to know up-front), you could update your application to restrict users from running them in the first place. 

Trisha

[Victor Voytovych]

Thank you Trisha,

I understand your point. The only question that remains open for me is how to limit query execution time from Java driver.
I think it's very important to have possibility to specify query execution timeout and be 100% sure that when timeout has been exceeded there is no operations related to the query running on server side.

Victor

[Victor Voytovych]

Any ideas?

[Jeff Yemin]

I think what you want is this: https://jira.mongodb.org/browse/SERVER-2212.

It's scheduled to be included in the next release of the MongoDB server.

Regards,

Jeff

Any ideas?

--
--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb
 
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Victor Voytovych]

Correct. Thank you.