Understand and interpret mongotop outputs?

[yongzh...@gmail.com]

Hi all,

I wrote a monitoring program which periodically invoke mongotop (without running parameter). I observed that extreme values of read mongotop reported may be up to 550K ms. Though the program will sleep 10 seconds to launch another invoking, it may take more time to wait when system is busy. Does the output means mongod spent 550 second on performing read operations on the top namespace in that invoking interval?

By the way, is there a way (argument) to specify the running times of mongotop, similar as "-n" option in mongostat tool? Thanks a lot.

Best Regards,
-Yongzhi

[Mike O'Brien]

Hi,

Can you give an example of the output where you see the extreme values?

If you want to programmatically access the data that mongotop returns, it may actually be easier to connect to the database directly from your program and use the "top" command, which is what the mongotop tool uses internally. See http://docs.mongodb.org/manual/reference/command/top/

Also, in 3.0+ the mongotop command does have a "-n" option, like mongostat.

[yongzh...@gmail.com]

Hi Mike,

Thank you so much for pointing out that mongotop actually internally use "top" command. I guess you meant execute `db.adminCommand("top")` at mongo shell prompt or equivalent operation. Those collections' total/readLock/writeLock numbers look accumulative. Therefore I guess mongotop at least conduct two more processing against top report: 1, calculate the spending time since last update; 2, order results from step #1 by total value then report the top 5. Is it right?

So top will report total usage statistics amount of time used for operating on each collection since the mongod is up. Those numbers will keep increasing until the reboot. Right? In my case, I want to take snapshot periodically to record database behavior. I try to improve the system performance by digging out the operation time distribution details. Do you think top command best fit my requirement?


Here is an example, I wrote a python tool to check CPU/memory usage of mongod, and gather/analyze mongostat&mongotop outputs. For convenience, I only dump the first collection record from mongotop report into the log. There is also a delay between mongostat & mongotop output processing. Thanks a lot.

Wed Mar 04 12:03:44 PDT 2015 INFO [PID=2533]: cpuUsage=100.90% memUsage=27.19GB insert=1 query=2 update=1397 delete=*0 total=35995ms read=35995ms write=0ms
Wed Mar 04 12:04:28 PDT 2015 INFO [PID=2533]: cpuUsage=100.90% memUsage=27.23GB insert=*0 query=1 update=22 delete=*0 total=42107ms read=42107ms write=0ms
Wed Mar 04 12:08:05 PDT 2015 INFO [PID=2533]: cpuUsage=101.00% memUsage=27.39GB insert=1 query=117 update=1162 delete=*0 total=515603ms read=515603ms write=0ms
Wed Mar 04 12:10:36 PDT 2015 INFO [PID=2533]: cpuUsage=100.90% memUsage=27.80GB insert=*0 query=*0 update=18 delete=*0 total=553650ms read=552976ms write=674ms
Wed Mar 04 12:11:07 PDT 2015 INFO [PID=2533]: cpuUsage=100.90% memUsage=27.89GB insert=*0 query=1 update=1580 delete=*0 total=56435ms read=55826ms write=609ms
Wed Mar 04 12:11:35 PDT 2015 INFO [PID=2533]: cpuUsage=100.00% memUsage=27.94GB insert=*0 query=26 update=1748 delete=*0 total=112079ms read=112079ms write=0ms
Wed Mar 04 12:11:49 PDT 2015 INFO [PID=2533]: cpuUsage=165.80% memUsage=28.01GB insert=1 query=2 update=491 delete=*0 total=4435ms read=4072ms write=363ms
Wed Mar 04 12:12:28 PDT 2015 INFO [PID=2533]: cpuUsage=179.80% memUsage=28.04GB insert=2 query=8 update=258 delete=*0 total=53509ms read=53139ms write=370ms
Wed Mar 04 12:13:23 PDT 2015 INFO [PID=2533]: cpuUsage=142.80% memUsage=28.13GB insert=1 query=1 update=2187 delete=*0 total=4078ms read=4078ms write=0ms
Wed Mar 04 12:14:40 PDT 2015 INFO [PID=2533]: cpuUsage=98.90% memUsage=28.28GB insert=*0 query=141 update=2043 delete=*0 total=251ms read=251ms write=0ms
Wed Mar 04 12:15:31 PDT 2015 INFO [PID=2533]: cpuUsage=98.90% memUsage=28.32GB insert=*0 query=2 update=24 delete=*0 total=36153ms read=36153ms write=0ms
Wed Mar 04 12:15:45 PDT 2015 INFO [PID=2533]: cpuUsage=163.90% memUsage=28.39GB insert=*0 query=*0 update=619 delete=*0 total=1283ms read=1283ms write=0ms

Best Regards,
-Yongzhi

[Mike O'Brien]

Yes, that's correct - mongotop takes samples from the top command at a polling interval, and then calculates the difference between each to calculate per-sample changes from the cumulative data. The numbers in the output of the "top" command will keep increasing untilt he next time the server is restarted.

[yongzh...@gmail.com]

Hi Mike,

Thank you so much for your confirmation. Can we use pymongo to invoke top command?

[Mike O'Brien]

Yes - here's an example:

from pymongo import MongoClient

admindb = MongoClient('localhost:27017').admin

top_data = admindb.command("top")

[yongzh...@gmail.com]

Cool. I really appreciate your prompt and great help! :-)