The group you are posting to is a
Usenet group . Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
From:
antituhan <dewangga.a... @merahciptamedia.co.id>
Date: Mon, 8 Oct 2012 22:02:09 -0700 (PDT)
Local: Tues, Oct 9 2012 1:02 am
Subject: High latency mongodb
Hello there, I have 1 server production, the spesification described below : - Intel(R) Xeon(R) CPU E5620 - RAM 16G
And my mongod version is : -- root@db:~# mongo --version MongoDB shell version: 2.0.2 root@db:~# --
The mongod can't serve data from the webserver, because the latency is unpredictable 'spike', before mongod has a spiked latency, this error occured. --- snip --- Tue Oct 9 11:11:42 [conn14110] warning: ClientCursor::yield can't unlock b/c of recursive lock ns: lintasberita.system.namespaces top: { opid: 174953454, active: true, lockType: "write", waitingForLock: false, secs_running: 0, op: "query", ns: "lintasberita", query: { $eval: CodeWScope( function (){ return db.getCollectionNames(); }, {}), args: {}
}, client: "192.168.1.5:36685", desc: "conn", threadId: "0x7efda4b9b700",
connectionId: 14110, numYields: 0 } Tue Oct 9 11:11:42 [conn14962] warning: ClientCursor::yield can't unlock b/c of recursive lock ns: lintasberita.system.namespaces top: { opid: 174953460, active: true, lockType: "write", waitingForLock: false, secs_running: 0, op: "query", ns: "lintasberita", query: { $eval: CodeWScope( function (){ return db.getCollectionNames(); }, {}), args: {} }, client: "192.168.1.3:58606", desc: "conn", threadId: "0x7efdc2ded700",
connectionId: 14962, numYields: 0 } Tue Oct 9 11:11:42 [conn14110] warning: ClientCursor::yield can't unlock b/c of recursive lock ns: lintasberita.system.namespaces top: { opid: 174953476, active: true, lockType: "write", waitingForLock: false, secs_running: 0, op: "query", ns: "lintasberita", query: { $eval: CodeWScope( function (){ return db.getCollectionNames(); }, {}), args: {} }, client: "192.168.1.5:36685", desc: "conn", threadId: "0x7efda4b9b700",
connectionId: 14110, numYields: 0 } --- snip --- Then, the log show this : --- snip --- Tue Oct 9 11:26:03 [conn20407] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn20407] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn20407] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 25662ms Tue Oct 9 11:26:03 [conn16211] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn16211] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn15263] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn15263] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn15263] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 143917ms Tue Oct 9 11:26:03 [conn20105] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn19641] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn20105] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn16590] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn17393] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn16590] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn17393] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn15303] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn17393] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 77848ms Tue Oct 9 11:26:03 [conn16211] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 132538ms Tue Oct 9 11:26:03 [conn15269] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn15303] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn15269] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn20064] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn20064] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn16590] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 126126ms Tue Oct 9 11:26:03 [conn20105] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 36561ms Tue Oct 9 11:26:03 [conn15269] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 115204ms Tue Oct 9 11:26:03 [conn15635] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn15303] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 115825ms Tue Oct 9 11:26:03 [conn16748] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn16748] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn15874] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn15635] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn15635] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 114401ms Tue Oct 9 11:26:03 [conn18505] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn19641] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn17248] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn16748] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 90472ms Tue Oct 9 11:26:03 [conn19641] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 50842ms Tue Oct 9 11:26:03 [conn20064] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 36559ms Tue Oct 9 11:26:03 [conn19537] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn19537] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn19537] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 52869ms Tue Oct 9 11:26:03 [conn14861] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn14861] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn17248] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn16839] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn17248] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 109893ms Tue Oct 9 11:26:03 [conn16839] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn15874] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn17252] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn16839] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 88452ms Tue Oct 9 11:26:03 [conn15874] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 138954ms Tue Oct 9 11:26:03 [conn17252] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn20210] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn14861] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 120642ms Tue Oct 9 11:26:03 [conn17252] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 80240ms Tue Oct 9 11:26:03 [conn20210] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn18505] ntoskip:0 ntoreturn:10 Tue Oct 9 11:26:03 [conn18308] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn18505] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 78555ms Tue Oct 9 11:26:03 [conn19944] assertion 11600 interrupted at shutdown ns:lintasberita.tag_list query:{ $query: { tag: /^Anti/u }, $orderby: { tag: 1 } } Tue Oct 9 11:26:03 [conn16560] assertion 11600 interrupted at shutdown ns:lintasberita.user_post query:{ $query: { tipe: "article", _deleted: false }, $orderby: { _last_update: -1 } } Tue Oct 9 11:26:03 [conn19944] ntoskip:0 ntoreturn:5 Tue Oct 9 11:26:03 [conn16560] ntoskip:0 ntoreturn:10 Tue Oct 9
...
read more »
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
antituhan <dewangga.a... @merahciptamedia.co.id>
Date: Tue, 9 Oct 2012 00:29:19 -0700 (PDT)
Local: Tues, Oct 9 2012 3:29 am
Subject: Re: High latency mongodb
Sorry, I forgot post the OS Details. -- root@db:~# uname -ar Linux db 2.6.32-24-server #43-Ubuntu SMP Thu Sep 16 16:05:42 UTC 2010 x86_64 GNU/Linux root@db:~# cat /etc/issue Ubuntu 10.04.1 LTS \n \l root@db:~# --
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
antituhan <dewangga.a... @merahciptamedia.co.id>
Date: Tue, 9 Oct 2012 01:41:26 -0700 (PDT)
Local: Tues, Oct 9 2012 4:41 am
Subject: Re: High latency mongodb
Myy server has been down again and the logs still showed this error : -- Tue Oct 9 15:40:48 [conn3497] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 89569ms Tue Oct 9 15:40:48 [conn3238] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 135740ms Tue Oct 9 15:40:48 [conn5970] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 85202ms Tue Oct 9 15:40:48 [conn3262] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 54901ms Tue Oct 9 15:40:48 [conn4235] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 25030ms Tue Oct 9 15:40:48 [conn6488] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 25889ms Tue Oct 9 15:40:48 [conn5914] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 56309ms Tue Oct 9 15:40:48 [conn4034] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 98019ms Tue Oct 9 15:40:48 [conn5461] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 107576ms Tue Oct 9 15:40:48 [conn7065] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 46518ms Tue Oct 9 15:40:48 [conn3591] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 117171ms Tue Oct 9 15:40:48 [conn5784] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 49793ms Tue Oct 9 15:40:48 [conn6201] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 75275ms Tue Oct 9 15:40:48 [conn5316] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 111841ms Tue Oct 9 15:40:48 [conn7419] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 1756ms Tue Oct 9 15:40:48 [conn4022] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 89012ms Tue Oct 9 15:40:48 [conn4968] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 139558ms Tue Oct 9 15:40:48 [conn3582] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 100754ms Tue Oct 9 15:40:48 [conn3208] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 106905ms Tue Oct 9 15:40:48 [conn5942] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 44990ms Tue Oct 9 15:40:48 [conn8624] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 10009ms Tue Oct 9 15:40:48 [conn6266] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 72999ms Tue Oct 9 15:40:48 [conn591] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 92211ms Tue Oct 9 15:40:48 [conn4334] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 138883ms Tue Oct 9 15:40:48 [conn5709] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 55786ms Tue Oct 9 15:40:48 [conn8674] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 8824ms Tue Oct 9 15:40:48 [conn8489] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 11781ms Tue Oct 9 15:40:48 [conn6919] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 50348ms Tue Oct 9 15:40:48 [conn4973] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 110054ms Tue Oct 9 15:40:48 [conn6113] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 47770ms Tue Oct 9 15:40:48 [conn6584] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 15253ms Tue Oct 9 15:40:48 [conn6539] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 25039ms Tue Oct 9 15:40:48 [conn6069] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 50159ms Tue Oct 9 15:40:48 [conn4790] query lintasberita.user_post ntoreturn:10 exception: interrupted at shutdown code:11600 reslen:69 99417ms --
You must
Sign in before you can post messages.
You do not have the permission required to post.
From:
David Hows <david.h... @10gen.com>
Date: Thu, 11 Oct 2012 22:34:44 -0700 (PDT)
Local: Fri, Oct 12 2012 1:34 am
Subject: Re: High latency mongodb
Hi Antituhan, Some of those queries look to have been running for quite a long time and that exception occurs when a global kill is issued.
Can you provide details what was happening on the server before these issues occurred?
Can you provide any details about the system to which those connections belong? You should be able to see the IP of the server which invoked the connection to determine.
Do you have MMS enabled or some other form of performance statistics available?
Cheers,
David
You must
Sign in before you can post messages.
You do not have the permission required to post.