Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
nodejs 100% cpu randomly
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  12 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Dor Tzur  
View profile  
 More options Nov 14 2012, 4:08 am
From: Dor Tzur <dor.t...@capriza.com>
Date: Wed, 14 Nov 2012 01:08:04 -0800 (PST)
Local: Wed, Nov 14 2012 4:08 am
Subject: nodejs 100% cpu randomly

We're having a problem where every once in a while one of our environments
our node app runs on 100% CPU. The server isn't very active and usually
runs on 0%-2% CPU.
I was wondering what are the common issues that might cause this problem
and what would be the best way to find out what causing this issue.

technical spec:
node version 0.8.14
ubuntu 11.10
Intel(R) Xeon(R) CPU E5645 @ 2.40GHz

gems used:
"express" : 2.5.x, "log" : "1.2.x", "redis" : "0.8.x", "socket.io" :
"0.9.x", "mongodb": ">= 0.9.6-7", "passport" : "0.x.x", "passport-local" :
"0.x.x",


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Chad Engler  
View profile  
 More options Nov 14 2012, 10:49 am
From: "Chad Engler" <Chad.Eng...@patlive.com>
Date: Wed, 14 Nov 2012 10:49:37 -0500
Local: Wed, Nov 14 2012 10:49 am
Subject: RE: [nodejs] nodejs 100% cpu randomly

Hehe, gems...

Sounds like garbage collection. Do you create large objects or buffers
temporarily?

-Chad

From: nodejs@googlegroups.com [mailto:nodejs@googlegroups.com] On Behalf
Of Dor Tzur
Sent: Wednesday, November 14, 2012 4:08 AM
To: nodejs@googlegroups.com
Subject: [nodejs] nodejs 100% cpu randomly

We're having a problem where every once in a while one of our
environments our node app runs on 100% CPU. The server isn't very active
and usually runs on 0%-2% CPU.
I was wondering what are the common issues that might cause this problem
and what would be the best way to find out what causing this issue.

technical spec:
node version 0.8.14
ubuntu 11.10
Intel(R) Xeon(R) CPU E5645 @ 2.40GHz

gems used:
"express" : 2.5.x, "log" : "1.2.x", "redis" : "0.8.x", "socket.io" :
"0.9.x", "mongodb": ">= 0.9.6-7", "passport" : "0.x.x", "passport-local"
: "0.x.x",

--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Anand George  
View profile  
 More options Nov 14 2012, 11:02 am
From: Anand George <mranandgeo...@gmail.com>
Date: Wed, 14 Nov 2012 21:31:53 +0530
Local: Wed, Nov 14 2012 11:01 am
Subject: Re: [nodejs] nodejs 100% cpu randomly

Are you using a service like forever? Had an API server running using
forever. Running without forever showed a CPU usage of about 5%. Running
with forever, the CPU usage would go to 100% on the first few calls. Using
another tool solved the issue.

On Wed, Nov 14, 2012 at 9:19 PM, Chad Engler <Chad.Eng...@patlive.com>wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Adam Crabtree  
View profile  
 More options Nov 14 2012, 11:40 am
From: Adam Crabtree <atcrabt...@gmail.com>
Date: Wed, 14 Nov 2012 08:40:05 -0800
Local: Wed, Nov 14 2012 11:40 am
Subject: Re: [nodejs] nodejs 100% cpu randomly

On the note of forever running at 100%, that likes to happen when your
process fails to start and forever attempts to restart it as fast as it can
die.

On Wed, Nov 14, 2012 at 8:01 AM, Anand George <mranandgeo...@gmail.com>wrote:

--
Better a little with righteousness
       than much gain with injustice.
Proverbs 16:8

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Michael Schoonmaker  
View profile  
 More options Nov 14 2012, 11:45 am
From: Michael Schoonmaker <michael.r.schoonma...@gmail.com>
Date: Wed, 14 Nov 2012 08:44:48 -0800
Local: Wed, Nov 14 2012 11:44 am
Subject: Re: [nodejs] nodejs 100% cpu randomly
... which is easily diagnosed with `forever list`, and can be assuaged
with forever's "spinning" parameters, "minUptime" and "spinSleepTime".

Other than Adam's reason, I've never seen CPU spikes because of
forever. Let's not have any straw men here.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Anand George  
View profile  
 More options Nov 14 2012, 11:56 am
From: Anand George <mranandgeo...@gmail.com>
Date: Wed, 14 Nov 2012 22:18:36 +0530
Local: Wed, Nov 14 2012 11:48 am
Subject: Re: [nodejs] nodejs 100% cpu randomly

The app has no issues running without forever... so not sure why forever
wants to restart the process. The app has been running for about 6 weeks
now using nohup with supervisor on AWS, have had at least six code changes
and have never faced this issue.

On Wed, Nov 14, 2012 at 10:10 PM, Adam Crabtree <atcrabt...@gmail.com>wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Jimb Esser  
View profile  
 More options Nov 14 2012, 12:50 pm
From: Jimb Esser <wastel...@gmail.com>
Date: Wed, 14 Nov 2012 09:50:53 -0800 (PST)
Local: Wed, Nov 14 2012 12:50 pm
Subject: Re: nodejs 100% cpu randomly

If it spikes to 100%, and then goes back down, I'd agree that sounds like
garbage collection.

Either way, the best way to know what is going on is to profile it!  I
think on current versions of node, node-webkit-agent[1] is currently the
best (only?) way to get a JavaScript CPU profile.  If it reports all of the
time in "(program)", then you may need a lower-level CPU profile to know
what's going on.

[1] https://github.com/c4milo/node-webkit-agent

  Jimb Esser


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Dor Tzur  
View profile  
 More options Nov 15 2012, 3:41 am
From: Dor Tzur <dor.t...@capriza.com>
Date: Thu, 15 Nov 2012 00:41:36 -0800 (PST)
Local: Thurs, Nov 15 2012 3:41 am
Subject: Re: nodejs 100% cpu randomly

A. We are not using forever so it's not relevant  to this case, thanks for
the input though.

B. We tried using nodetime but it couldn't profile the cpu when it's at
100%. Do you think node-web-kit has a better chance?

C. We've been using MongoDB without defining safe mode and with
auto-reconnect turned on. do you think one of these options might cause
this?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Anton Whalley  
View profile  
 More options Nov 15 2012, 4:41 am
From: Anton Whalley <anton.whal...@nearform.com>
Date: Thu, 15 Nov 2012 01:41:09 -0800 (PST)
Local: Thurs, Nov 15 2012 4:41 am
Subject: Re: nodejs 100% cpu randomly

Can you put it on an illumOS based system temporarily and get DTracing?
At the very least this would tell you if its infrastructure or application.

Spin up a Joyent SmartOS or do a local install of OpenIndiana.
Then install your app

This d script will show you when garbage collection is starting and how
long it takes.

node*:::gc-start
{
    self->ts = timestamp;

}

node*:::gc-done
{
    printf("Garbage Collection - [%Y] - %d",
              walltimestamp,
              timestamp - self->ts);
              self->ts = 0

}

Anton


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Ben Noordhuis  
View profile  
 More options Nov 15 2012, 9:42 am
From: Ben Noordhuis <i...@bnoordhuis.nl>
Date: Thu, 15 Nov 2012 15:42:31 +0100
Local: Thurs, Nov 15 2012 9:42 am
Subject: Re: [nodejs] Re: nodejs 100% cpu randomly
On Thu, Nov 15, 2012 at 10:41 AM, Anton Whalley

In that vein, if you're on a semi-recent RHEL-based system (CentOS,
Fedora), you can use this systemtap script.  It prints aggregated GC
statistics every 5 seconds and at program exit.  Run with `sudo stap
gc.stp -c 'node script.js'`.

Note that systemtap support currently only exists in master.
Debian/Ubuntu probably won't work, the systemtap in the official
repositories is too old.

#!/usr/bin/env stap

global samples
global all_samples
global timestamp

probe process("node").mark("gc__start")
{
  timestamp = gettimeofday_us()

}

probe process("node").mark("gc__done")
{
  sample = gettimeofday_us() - timestamp
  samples <<< sample
  all_samples <<< sample

}

probe timer.s(5)
{
  print(@hist_log(samples))
  printf("min:%d avg:%d max:%d count:%d\n",
         @min(samples), @avg(samples),
         @max(samples), @count(samples))
  delete samples

}

probe end
{
  print(@hist_log(all_samples))
  printf("min:%d avg:%d max:%d count:%d\n",
         @min(all_samples), @avg(all_samples),
         @max(all_samples), @count(all_samples))

}

As a gist: https://gist.github.com/4078925

 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
darcy  
View profile  
 More options Nov 15 2012, 9:33 pm
From: darcy <freeda...@gmail.com>
Date: Thu, 15 Nov 2012 18:33:34 -0800 (PST)
Local: Thurs, Nov 15 2012 9:33 pm
Subject: Re: nodejs 100% cpu randomly

node-webkit-agent is a good tool.
if you just want to know whether it's caused by gc, the easiest way is run
your program by "node --trace_gc myapp.js", and you can see GC time like
this:

  785292 ms: Mark-sweep 9.5 (46.0) -> 9.4 (46.0) MB, 14 ms ...

where '14ms' means busy doing GC with CPU usage of nearly 100%.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Murvin Lai  
View profile  
 More options Nov 16 2012, 5:51 pm
From: Murvin Lai <murvin...@gmail.com>
Date: Fri, 16 Nov 2012 14:51:31 -0800
Local: Fri, Nov 16 2012 5:51 pm
Subject: Re: [nodejs] Re: nodejs 100% cpu randomly

For your server, how many RAM u have?   it may be related to memory.
try either increase your physical memory, or run node with extra memory
allocated.  (default is 1GB)


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »