Received: by 10.43.130.199 with SMTP id hn7mr1209824icc.24.1352990572325;
        Thu, 15 Nov 2012 06:42:52 -0800 (PST)
X-BeenThere: nodejs@googlegroups.com
Received: by 10.50.140.40 with SMTP id rd8ls46010igb.2.canary; Thu, 15 Nov
 2012 06:42:32 -0800 (PST)
Received: by 10.66.85.133 with SMTP id h5mr206400paz.23.1352990552336;
        Thu, 15 Nov 2012 06:42:32 -0800 (PST)
Received: by 10.66.85.133 with SMTP id h5mr206399paz.23.1352990552327;
        Thu, 15 Nov 2012 06:42:32 -0800 (PST)
Return-Path: <i...@bnoordhuis.nl>
Received: from mail-pa0-f51.google.com (mail-pa0-f51.google.com [209.85.220.51])
        by gmr-mx.google.com with ESMTPS id r4si3268785paz.1.2012.11.15.06.42.32
        (version=TLSv1/SSLv3 cipher=OTHER);
        Thu, 15 Nov 2012 06:42:32 -0800 (PST)
Received-SPF: pass (google.com: domain of i...@bnoordhuis.nl designates 209.85.220.51 as permitted sender) client-ip=209.85.220.51;
Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of i...@bnoordhuis.nl designates 209.85.220.51 as permitted sender) smtp.mail=i...@bnoordhuis.nl
Received: by mail-pa0-f51.google.com with SMTP id kq12so1018958pab.38
        for <nodejs@googlegroups.com>; Thu, 15 Nov 2012 06:42:32 -0800 (PST)
        d=google.com; s=20120113;
        h=mime-version:x-originating-ip:in-reply-to:references:date
         :message-id:subject:from:to:content-type:x-gm-message-state;
        bh=8/D8r8q2X4PiGkurRH2lTzfmXaRe8Hw5QRbIlXzw614=;
        b=OyQ8NaCZ29t5JKBINy8YjxRoN0yWpKvm1OMZ/itzmtxUKHV/h7VulwoAzbD5DfoUio
         nGC/nxYA1Fe08Owp/4z+mmqa4MGYLTDcFeVyr4fTruVEnmbbYc+AS0cLdbZkWJSPlGN6
         6StmcNV4kNWrVllRAdzGn2KOruz1xGnCHAuN+eZ4TWJOSKmjKklLar24xXx10eIOWP64
         wwz2jR5xwGab9kvFWfwcoUoiY9wIMr1cvy8hpWzKsN2RnTFxYBzV4z3ZCSI4QBJZ0vWX
         kD4uggl9hVC5q45jglCSy7DDDD2fWf1r2GLvC0KT7bLt+Dj2p88tl7D0yPhdrMf0RKWN
         C+yQ==
MIME-Version: 1.0
Received: by 10.68.248.1 with SMTP id yi1mr5483040pbc.93.1352990552025; Thu,
 15 Nov 2012 06:42:32 -0800 (PST)
Received: by 10.68.209.104 with HTTP; Thu, 15 Nov 2012 06:42:31 -0800 (PST)
X-Originating-IP: [87.214.96.125]
In-Reply-To: <13f6b87a-1cd1-48b7-87c4-454c98088b5c@googlegroups.com>
References: <a070c827-b3f3-4820-af7a-1c0fe8880682@googlegroups.com>
	<13f6b87a-1cd1-48b7-87c4-454c98088b5c@googlegroups.com>
Date: Thu, 15 Nov 2012 15:42:31 +0100
Message-ID: <CAHQurc8RDPOB60TWp898HFVtv4PmRCWS4-8uh275NB7pC1P...@mail.gmail.com>
Subject: Re: [nodejs] Re: nodejs 100% cpu randomly
From: Ben Noordhuis <i...@bnoordhuis.nl>
To: nodejs@googlegroups.com
Content-Type: text/plain; charset=ISO-8859-1
X-Gm-Message-State: ALoCoQnxxf3E0hXB4UBma/9QXGhAV6gc++TaZdB/+4koSStupELnzMhkknBzVErvzVS1RBardJD+

On Thu, Nov 15, 2012 at 10:41 AM, Anton Whalley
<anton.whal...@nearform.com> wrote:
> Can you put it on an illumOS based system temporarily and get DTracing?
> At the very least this would tell you if its infrastructure or application.
>
> Spin up a Joyent SmartOS or do a local install of OpenIndiana.
> Then install your app
>
> This d script will show you when garbage collection is starting and how long
> it takes.
>
>
> node*:::gc-start
> {
>     self->ts = timestamp;
> }
>
> node*:::gc-done
> {
>     printf("Garbage Collection - [%Y] - %d",
>               walltimestamp,
>               timestamp - self->ts);
>               self->ts = 0
> }

In that vein, if you're on a semi-recent RHEL-based system (CentOS,
Fedora), you can use this systemtap script.  It prints aggregated GC
statistics every 5 seconds and at program exit.  Run with `sudo stap
gc.stp -c 'node script.js'`.

Note that systemtap support currently only exists in master.
Debian/Ubuntu probably won't work, the systemtap in the official
repositories is too old.

#!/usr/bin/env stap

global samples
global all_samples
global timestamp

probe process("node").mark("gc__start")
{
  timestamp = gettimeofday_us()
}

probe process("node").mark("gc__done")
{
  sample = gettimeofday_us() - timestamp
  samples <<< sample
  all_samples <<< sample
}

probe timer.s(5)
{
  print(@hist_log(samples))
  printf("min:%d avg:%d max:%d count:%d\n",
         @min(samples), @avg(samples),
         @max(samples), @count(samples))
  delete samples
}

probe end
{
  print(@hist_log(all_samples))
  printf("min:%d avg:%d max:%d count:%d\n",
         @min(all_samples), @avg(all_samples),
         @max(all_samples), @count(all_samples))
}

As a gist: https://gist.github.com/4078925