Realizing node is still a nascent project, Ryan's away from the list
for awhile, and node vs. thin is an apple vs. oranges sort of thing,
just sharing recent observations here on the chance others might find
informative...
I ran ab -c100 -n3000000 against the node and thin (ruby) web servers
to watch them each from linux's top display. While thin kept its
memory allocation relatively conservative and constant (~6.5MB) over
the duration of the run (as viewed in top's DATA column over the time
it serviced requests), node's started out at 33MB and gradually
climbed up to 353MB by the time all three million requests had been
processed. I haven't done so yet but believe if I increase the number
of requests sent to node and wait long enough, it will eventually fall
over where v8's heap.cc and assembler-ia32.h define the limit, 512MB.
I'd be interested to learn what others see on their own systems. This
was Ubuntu 8.10 Desktop running inside VirtualBox on Mac OS Leopard.
> Realizing node is still a nascent project, Ryan's away from the list
> for awhile, and node vs. thin is an apple vs. oranges sort of thing,
> just sharing recent observations here on the chance others might find
> informative...
> I ran ab -c100 -n3000000 against the node and thin (ruby) web servers
> to watch them each from linux's top display. While thin kept its
> memory allocation relatively conservative and constant (~6.5MB) over
> the duration of the run (as viewed in top's DATA column over the time
> it serviced requests), node's started out at 33MB and gradually
> climbed up to 353MB by the time all three million requests had been
> processed. I haven't done so yet but believe if I increase the number
> of requests sent to node and wait long enough, it will eventually fall
> over where v8's heap.cc and assembler-ia32.h define the limit, 512MB.
> I'd be interested to learn what others see on their own systems. This
> was Ubuntu 8.10 Desktop running inside VirtualBox on Mac OS Leopard.
That is, keep-alives were used. This finished the run and ** memory
went back down **.
So,
- loading node test_server.js ... 12MB resident
- during run of 'ab' ... 286MB resident (ouch)
- after 'ab' finishes all 2.5M requests ... 116MB resident (why not
12MB again?)
I fixed one problem which was related to a missing HandleScope in the event processing code. Fixed in 50c0d16208aab392d815e420f3423d23283a31de.
It's very hard to spot these errors and I wonder if there is some V8 technique that I'm missing to catch these errors. There seems to still be a smaller leak somewhere else.
Requests per second: 3515.67 [#/sec] (mean)
Failed requests: 0
Transfer rate: 391.39 [Kbytes/sec] received
Watching 'top' showed node using 34-35m resident memory. It does not
grow. Good stuff.
Regarding the missing HandleScope. I'm not intimately familiar with
the V8 API. However, I whipped up this little Perl script to
**maybe** find other cases like this in the Node V8 code. YMMV.
It found one thing that may or may not be a problem:
http.cc:153 - no HandleScope found in function
Function started at line 136
Concern is from line content: return Local<String>();
static inline Local<String>
GetMethod (int method)
{
switch (method) {
case HTTP_COPY: return String::NewSymbol("COPY");
case HTTP_DELETE: return String::NewSymbol("DELETE");
case HTTP_GET: return String::NewSymbol("GET");
case HTTP_HEAD: return String::NewSymbol("HEAD");
case HTTP_LOCK: return String::NewSymbol("LOCK");
case HTTP_MKCOL: return String::NewSymbol("MKCOL");
case HTTP_MOVE: return String::NewSymbol("MOVE");
case HTTP_OPTIONS: return String::NewSymbol("OPTIONS");
case HTTP_POST: return String::NewSymbol("POST");
case HTTP_PROPFIND: return String::NewSymbol("PROPFIND");
case HTTP_PROPPATCH: return String::NewSymbol("PROPPATCH");
case HTTP_PUT: return String::NewSymbol("PUT");
case HTTP_TRACE: return String::NewSymbol("TRACE");
case HTTP_UNLOCK: return String::NewSymbol("UNLOCK");
}
return Local<String>();
}
Again, I'm not really sure if you need a HandleScope here. I suppose
it's unclear to me if the HandleScope in the caller of GetMethod is
"good enough" to grab the returned Local. Not sure.
Here's the script. Run it from the node/src dir like: $ perl
find_handle_scope_issues.pl
#!/usr/bin/perl
use strict;
foreach my $filename (<*.cc>) {
local $/;
open(FH, $filename) or die "$filename: $!";
my @lines = split /\n/, <FH>;
close(FH);
my $most_recent_func_start_line = 0;
my $most_recent_handle_scope_line = 0;
my $line_no = 0;
if ( $line =~ /\b(?:Local|Handle)\b/ ) {
if ( $most_recent_handle_scope_line <
$most_recent_func_start_line and
$most_recent_func_start_line > 0 ) {
warn "$filename:$line_no - no HandleScope found in function
\n".
"\tFunction started at line $most_recent_func_start_line
\n".
"\tConcern is from line content: $line\n";
}
}
}
}
On Jul 24, 9:20 am, ryan dahl <coldredle...@gmail.com> wrote:
> I fixed one problem which was related to a missing HandleScope in the
> event processing code. Fixed in
> 50c0d16208aab392d815e420f3423d23283a31de.
> It's very hard to spot these errors and I wonder if there is some V8
> technique that I'm missing to catch these errors. There seems to still
> be a smaller leak somewhere else.
> Requests per second: 3515.67 [#/sec] (mean)
> Failed requests: 0
> Transfer rate: 391.39 [Kbytes/sec] received
> Watching 'top' showed node using 34-35m resident memory. It does not
> grow. Good stuff.
> Regarding the missing HandleScope. I'm not intimately familiar with
> the V8 API. However, I whipped up this little Perl script to
> **maybe** find other cases like this in the Node V8 code. YMMV.
> It found one thing that may or may not be a problem:
> http.cc:153 - no HandleScope found in function
> Function started at line 136
> Concern is from line content: return Local<String>();
> static inline Local<String>
> GetMethod (int method)
> {
> switch (method) {
> case HTTP_COPY: return String::NewSymbol("COPY");
> case HTTP_DELETE: return String::NewSymbol("DELETE");
> case HTTP_GET: return String::NewSymbol("GET");
> case HTTP_HEAD: return String::NewSymbol("HEAD");
> case HTTP_LOCK: return String::NewSymbol("LOCK");
> case HTTP_MKCOL: return String::NewSymbol("MKCOL");
> case HTTP_MOVE: return String::NewSymbol("MOVE");
> case HTTP_OPTIONS: return String::NewSymbol("OPTIONS");
> case HTTP_POST: return String::NewSymbol("POST");
> case HTTP_PROPFIND: return String::NewSymbol("PROPFIND");
> case HTTP_PROPPATCH: return String::NewSymbol("PROPPATCH");
> case HTTP_PUT: return String::NewSymbol("PUT");
> case HTTP_TRACE: return String::NewSymbol("TRACE");
> case HTTP_UNLOCK: return String::NewSymbol("UNLOCK");
> }
> return Local<String>();
> }
> Again, I'm not really sure if you need a HandleScope here. I suppose
> it's unclear to me if the HandleScope in the caller of GetMethod is
> "good enough" to grab the returned Local. Not sure.
> Here's the script. Run it from the node/src dir like: $ perl
> find_handle_scope_issues.pl
> #!/usr/bin/perl
> use strict;
> foreach my $filename (<*.cc>) {
> local $/;
> open(FH, $filename) or die "$filename: $!";
> my @lines = split /\n/, <FH>;
> close(FH);
> my $most_recent_func_start_line = 0;
> my $most_recent_handle_scope_line = 0;
> my $line_no = 0;
> if ( $line =~ /\b(?:Local|Handle)\b/ ) {
> if ( $most_recent_handle_scope_line <
> $most_recent_func_start_line and
> $most_recent_func_start_line > 0 ) {
> warn "$filename:$line_no - no HandleScope found in function
> \n".
> "\tFunction started at line $most_recent_func_start_line
> \n".
> "\tConcern is from line content: $line\n";
> }
> }
> }
> }
> On Jul 24, 9:20 am, ryan dahl <coldredle...@gmail.com> wrote:
> > I fixed one problem which was related to a missing HandleScope in the
> > event processing code. Fixed in
> > 50c0d16208aab392d815e420f3423d23283a31de.
> > It's very hard to spot these errors and I wonder if there is some V8
> > technique that I'm missing to catch these errors. There seems to still
> > be a smaller leak somewhere else.
