Excessive memory usage ratio - nodejs

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion Excessive memory usage ratio
Date: Wed, 1 Aug 2012 12:04:26 -0700 (PDT)
From: Jimb Esser <wastel...@gmail.com>
To: nodejs@googlegroups.com
Message-Id: <09000899-f1a5-46e4-a95a-4cf1e4c74e0a@googlegroups.com>
In-Reply-To: <738e2b8d-607d-4a15-a102-ba5a737589e9@googlegroups.com>
References: <834b5038-16ff-4e63-ae66-524bc42d222d@googlegroups.com>
 <CAAPeUNk4POdiNVKXUEhxFCh32oz5jyupYuQAXWJy9dDLiHMAaQ@mail.gmail.com>
 <738e2b8d-607d-4a15-a102-ba5a737589e9@googlegroups.com>
Subject: Re: [nodejs] Excessive memory usage ratio
MIME-Version: 1.0
Content-Type: multipart/mixed; 
	boundary="----=_Part_1570_24149474.1343847866528"

------=_Part_1570_24149474.1343847866528
Content-Type: multipart/alternative; 
	boundary="----=_Part_1571_5804961.1343847866528"

------=_Part_1571_5804961.1343847866528
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

You may want to try mtrace, I've exposed it in a node module here:
https://github.com/Jimbly/node-mtrace

It allows you to trace and analyze native heap memory usage, though if most 
of the native code you're in is C++, it's just going to say all the memory 
was allocated by "operator new()", but it's worth a shot.


On Wednesday, August 1, 2012 12:23:33 AM UTC-7, Rusty wrote:
>
> On Monday, July 30, 2012 6:55:23 PM UTC-7, Marak Squires wrote:
>>
>> It's possible you've got a leak in your application code ( accidentally 
>> not calling .end on a response stream is a common issue I've seen ).
>
>
> Thanks for your help Marak.
>
> The code uses socket.io, and in my testing I have it serving only 8 
> clients so the number of connections opened should be constant.  I am not 
> opening any connections myself beyond the basic usage.
>
> I will work on a test case to narrow things down.
>  
>
>> There are profiling tools to help track this sort of thing down, but I'm 
>> not sure which is the best these days. Usually, I find that a code review 
>> is enough to track down this sort of thing.
>>
>
> Code review is great, and this is all after a thorough review and cleanup. 
>  However, it seems like a lack of tools for this situation is a significant 
> drawback to this platform.
>
> All of the memory tools I am aware of, I have tried.  All the V8 heap 
> tools are of little use since the JS heap isn't growing.  The distribution 
> of types across the heap is relatively constant as well (as far as I can 
> see).
>
> Valgrind is great for "real" leaks in "native" code, but it doesn't really 
> help here.
>  
>
>> It's also possible there is a leak in core, or in one of the third-party 
>> modules you are using. Further investigation seems warranted.
>>
>   
> Hopefully I can come up with a reduction that explains what is going on. 
> :-/
>
> The more problems I find like this, the more I realize why people enjoy 
> there silly "stable" environments. ;-)
>
> Thanks again Marak.
>
> ~Rusty
>

------=_Part_1571_5804961.1343847866528
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

You may want to try mtrace, I've exposed it in a node module here:<div><a h=
ref=3D"https://github.com/Jimbly/node-mtrace">https://github.com/Jimbly/nod=
e-mtrace</a></div><div><br></div><div>It allows you to trace and analyze na=
tive heap memory usage, though if most of the native code you're in is C++,=
 it's just going to say all the memory was allocated by "operator new()", b=
ut it's worth a shot.</div><div><br></div><br>On Wednesday, August 1, 2012 =
12:23:33 AM UTC-7, Rusty wrote:<blockquote class=3D"gmail_quote" style=3D"m=
argin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"=
>On Monday, July 30, 2012 6:55:23 PM UTC-7, Marak Squires wrote:<blockquote=
 class=3D"gmail_quote" style=3D"margin:0;margin-left:0.8ex;border-left:1px =
#ccc solid;padding-left:1ex">It's possible you've got a leak in your applic=
ation code ( accidentally not calling .end on a response stream is a common=
 issue I've seen ).</blockquote><div><br></div><div>Thanks for your help Ma=
rak.</div><div><br></div><div>The code uses <a href=3D"http://socket.io" ta=
rget=3D"_blank">socket.io</a>, and in my testing I have it serving only 8 c=
lients so the number of connections opened should be constant. &nbsp;I am n=
ot opening any connections myself beyond the basic usage.<br></div><div><br=
>I will work on a test case to narrow things down.<br></div><div>&nbsp;</di=
v><blockquote class=3D"gmail_quote" style=3D"margin:0;margin-left:0.8ex;bor=
der-left:1px #ccc solid;padding-left:1ex"><div>There are profiling tools to=
 help track this sort of thing down, but I'm not sure which is the best the=
se days. Usually, I find that a code review is enough to track down this so=
rt of thing.<br></div></blockquote><div><br></div><div>Code review is great=
, and this is all after a thorough review and cleanup. &nbsp;However, it se=
ems like a lack of tools for this situation is a significant drawback to th=
is platform.</div><div><br></div><div>All of the memory tools I am aware of=
, I have tried. &nbsp;All the V8 heap tools are of little use since the JS =
heap isn't growing. &nbsp;The distribution of types across the heap is rela=
tively constant as well (as far as I can see).</div><div><br></div><div>Val=
grind is great for "real" leaks in "native" code, but it doesn't really hel=
p here.</div><div>&nbsp;</div><blockquote class=3D"gmail_quote" style=3D"ma=
rgin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex"><div>=
</div><div>
<div>It's also possible there is a leak in core, or in one of the third-par=
ty modules you are using. Further investigation seems warranted.<br></div><=
/div></blockquote><div>&nbsp;&nbsp;<br></div><div>Hopefully I can come up w=
ith a reduction that explains what is going on. :-/</div><div><br></div><di=
v>The more problems I find like this, the more I realize why people enjoy t=
here silly "stable" environments. ;-)</div><div><br></div><div>Thanks again=
 Marak.</div><div><br></div><div>~Rusty</div></blockquote>
------=_Part_1571_5804961.1343847866528--

------=_Part_1570_24149474.1343847866528--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
