I have a node program using express, socket.io, and amqp.
It seems to grow in memory usage very quick under load. By the time it reaches about 25,000 events complete, process.memoryUsage() reports the following. { rss: 1072037888, heapTotal: 54075392, heapUsed: 30898800 }
That is about 1GB of RSS and 54MB of total heap. The process is almost 20x the heap size!
Does that scare anyone else? :-/
During the time it is growing, the JS heap usage fluctuates between 10MB and 42MB, depending on the number of clients connected and the type of events being processed.
I am currently running v0.8.4, but I saw the same problem on v0.8.2 and v0.8.3.
It's possible you've got a leak in your application code ( accidentally not
calling .end on a response stream is a common issue I've seen ).
There are profiling tools to help track this sort of thing down, but I'm
not sure which is the best these days. Usually, I find that a code review
is enough to track down this sort of thing.
It's also possible there is a leak in core, or in one of the third-party
modules you are using. Further investigation seems warranted.
On Mon, Jul 30, 2012 at 12:44 PM, Rusty <gic...@gmail.com> wrote:
> Howdy folks,
> I have a node program using express, socket.io, and amqp.
> It seems to grow in memory usage very quick under load. By the time it
> reaches about 25,000 events complete, process.memoryUsage() reports the
> following.
> { rss: 1072037888, heapTotal: 54075392, heapUsed: 30898800 }
> That is about 1GB of RSS and 54MB of total heap. The process is almost
> 20x the heap size!
> Does that scare anyone else? :-/
> During the time it is growing, the JS heap usage fluctuates between 10MB
> and 42MB, depending on the number of clients connected and the type of
> events being processed.
> I am currently running v0.8.4, but I saw the same problem on v0.8.2
> and v0.8.3.
On Monday, July 30, 2012 6:55:23 PM UTC-7, Marak Squires wrote:
> It's possible you've got a leak in your application code ( accidentally > not calling .end on a response stream is a common issue I've seen ).
Thanks for your help Marak.
The code uses socket.io, and in my testing I have it serving only 8 clients so the number of connections opened should be constant. I am not opening any connections myself beyond the basic usage.
I will work on a test case to narrow things down.
> There are profiling tools to help track this sort of thing down, but I'm > not sure which is the best these days. Usually, I find that a code review > is enough to track down this sort of thing.
Code review is great, and this is all after a thorough review and cleanup. However, it seems like a lack of tools for this situation is a significant drawback to this platform.
All of the memory tools I am aware of, I have tried. All the V8 heap tools are of little use since the JS heap isn't growing. The distribution of types across the heap is relatively constant as well (as far as I can see).
Valgrind is great for "real" leaks in "native" code, but it doesn't really help here.
> It's also possible there is a leak in core, or in one of the third-party > modules you are using. Further investigation seems warranted.
Hopefully I can come up with a reduction that explains what is going on. :-/
The more problems I find like this, the more I realize why people enjoy there silly "stable" environments. ;-)
It allows you to trace and analyze native heap memory usage, though if most of the native code you're in is C++, it's just going to say all the memory was allocated by "operator new()", but it's worth a shot.
On Wednesday, August 1, 2012 12:23:33 AM UTC-7, Rusty wrote:
> On Monday, July 30, 2012 6:55:23 PM UTC-7, Marak Squires wrote:
>> It's possible you've got a leak in your application code ( accidentally >> not calling .end on a response stream is a common issue I've seen ).
> Thanks for your help Marak.
> The code uses socket.io, and in my testing I have it serving only 8 > clients so the number of connections opened should be constant. I am not > opening any connections myself beyond the basic usage.
> I will work on a test case to narrow things down.
>> There are profiling tools to help track this sort of thing down, but I'm >> not sure which is the best these days. Usually, I find that a code review >> is enough to track down this sort of thing.
> Code review is great, and this is all after a thorough review and cleanup. > However, it seems like a lack of tools for this situation is a significant > drawback to this platform.
> All of the memory tools I am aware of, I have tried. All the V8 heap > tools are of little use since the JS heap isn't growing. The distribution > of types across the heap is relatively constant as well (as far as I can > see).
> Valgrind is great for "real" leaks in "native" code, but it doesn't really > help here.
>> It's also possible there is a leak in core, or in one of the third-party >> modules you are using. Further investigation seems warranted.
> Hopefully I can come up with a reduction that explains what is going on. > :-/
> The more problems I find like this, the more I realize why people enjoy > there silly "stable" environments. ;-)
I had a similar problem, with a simple parser that just reads the twitter stream. The initial reading rate declined the longer the process was running while the memory usage grew and grew. The old parser used String.slice(). I rewrote it to use String.split to cut the single JSON objects from the stream API. I am currently running a long test run. Now the rate is climbing (slowly, but climbing) and memory usage is stable
Does someone here know if that is a known problem for node 8.4? So far I did not find anything in the issue list on GitHub. I can provide a test case to reproduce this.
