pymongo using mongo shell commands

[Jason]

Hi. I recently started using Mongo, and love it so far. Everything has
been very easy to pick up, and I am just now getting to my first
roadbump. I am trying to write a python script to monitor some aspects
of my mongodb instance, and keep track of various things.

A specific piece of information I want is how chunks are distributed
to the shards that I have through time. I know that from the mongo
shell, the db.printShardingStatus() command will return how many
chunks are on each shard, which is exactly what I am looking for.
However, I am unsure of how to get this information programmatically,
through pymongo.

For other mongo shell commands, such as the database stats command
"db.stats()" or "db.serverStatus()", I can get the information
programmatically through python by calling
"database.command("dbstats")" or "database.command("serverStatus"),
where database is a pymongo database instance. However, there doesnt
seem to be a similar keyword for the "db.printShardingStatus()" shell
command... at least not that I can find.

So, my question is:
a) is there such a keyword I can pass to the database.command()
function, like "dbstats" to accomplish this task programmatically, and
if so what is it
b) if not, what is an easy way of doing what I want... namely,
getting the number of chunks on each shard at particular points in
time programmatically


Thanks,
-Jason

[Nat Luengnaruemitchai]

Take a look at some samples from fangofmongo. You should be able to jumpstart yourself.

https://github.com/Fiedzia/Fang-of-Mongo/blob/fom_object/fangofmongo/fom/views.py

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

[sridhar]

There is no db command for db.printShardingStatus() but if you run
printShardingStatus (without the paranthesis) from the javascript
shell you can see what it is doing. http://www.mongodb.org/display/DOCS/Sharding+Config+Schema
defines the collections in the config database and this is what
db.printShardingStatus is reporting on. The chunks collection has
information about the chunks

On Jun 29, 5:24 pm, Nat Luengnaruemitchai <nat.lu...@gmail.com> wrote:
> Take a look at some samples from fangofmongo. You should be able to

> jumpstart yourself.https://github.com/Fiedzia/Fang-of-Mongo/blob/fom_object/fangofmongo/...

[Jason D'Souza]

Thanks for the replys guys. 

So, I typed in printShardingStatus, without the parenthesis, and looked over the javascript code that it displayed, and commented out parts that I don't need for my purposes: https://gist.github.com/1056481. 

This was helpful, but unfortunately I am not comfortable enough with the code to convert it to a usable python function. After looking at the javascript, it seems that I need to iterate through all the documents in the chunks database, and sort/ sum them somehow, but I'm not sure how. For instance, lines 33-46 of the gist seem to do something like that, but I am unclear as to what it does (looks like some sort of map reduce or something). 

Can someone who better understands what the javascript is doing, and who is comfortable with python either explain the javascript code, or do the conversion to python so I can better understand? Or both :)

[Nat]

You don't have to follow it completely. You can browse around system collections like in javascript code. It should give you better understanding. That part is just to count number of chunks for each shard.

[Erez Zarum]

i wrote a small snippet to do that, enjoy.

 #!/usr/bin/env python
 #
 # This code is based on printShardingStatus function from mongodb engine.
 #
 # Erez Zarum, <erezz at icinga.org.il>, 2011
 #
 
 import re
 import sys
 import json
 from pymongo import Connection, errors
 
 host = 'localhost' # mongos host
 port = 27017 # mongos port
 
 try:
     session = Connection(host=host, port=port)
 except errors.ConnectionFailure:
     print 'Failed to connect to: %s:%d' % (host, port)
     sys.exit(1)
 
 config = session['config']
 databases = config.databases.find( { "partitioned" : True } )
 
 for db in databases:
     print db['_id']
     collections = config.collections.find( { "_id" : { "$regex" : re.compile("^" + db['_id'] + ".") } }, { "dropped" : False } )
     for colls in collections:
         print '\t' + colls['_id'] + ' chunks:'
         result = config.chunks.group( key=['shard'],
                                       condition={ 'ns' : colls['_id'] },
                                       initial={ "nChunks" : 0 },
                                       reduce="function (doc, out) { out.nChunks++; }"
                                     )  
         for value in result:
             print '\t\t' + value['shard'] + ' ' + str(value['nChunks'])
     print

--

You received this message because you are subscribed to the Google Groups "mongodb-user" group.

To view this discussion on the web visit https://groups.google.com/d/msg/mongodb-user/-/hOkcExYvl4gJ.

[Erez Zarum]

you can ignore the "import json" part.

[Jason D'Souza]

Infin1ty, thanks a lot. This is exactly what I needed.