[libxmljs] node & memory management

[billywhizz]

folks,

i just came accross this strange function in the node.js 0.1.90
release (look in node_buffer.cc - http://github.com/ry/node/blob/v0.1.91/src/node_buffer.cc#L60).

v8::AdjustAmountOfExternalAllocatedMemory

I then found this article:
http://create.tpsitulsa.com/wiki/V8/Garbage_Collection

i had a scan through the libxmljs code and it doesn't seem to use
this. it might be worth looking into if the garbage collection/memory
leaks are still an issue...


--
Subscription settings: http://groups.google.com/group/libxmljs/subscribe?hl=en

[Marco Rogers]

I had a look at this. As usual, the documentation on v8 is sparse.
But as far as i can tell this is useful when memory is allocated that
lives outside of the v8 heap space. When that happens you should let
v8 know about it so that garbage collector can be smarter about when
to collect. That's what this function does.

see: http://create.tpsitulsa.com/wiki/V8/Garbage_Collection

I'm not sure if or where we need this in libxmljs. I'll have to take
a deeper look soon. But in the meantime it doesn't actually cause
memory leaks. It just means you could allocate a lot of memory before
v8 decides to run the GC.

Thanks, still got a lot to learn
:Marco

On Apr 18, 7:32 am, billywhizz <apjohn...@gmail.com> wrote:
> folks,
>
> i just came accross this strange function in the node.js 0.1.90

> release (look in node_buffer.cc -http://github.com/ry/node/blob/v0.1.91/src/node_buffer.cc#L60).

[billywhizz]

Marco,

if you have a look at the code for node_buffer in the latest releases
of node.js, it uses this function when allocation/deallocating blobs
of memory. i have tried my damndest to get an out of memory exception
using buffers but it never seems to have a problem. am wondering could
this function be why?

here's a script that never crashes and is doing lots of heavy buffer
allocations in a very tight loop...

http://gist.github.com/374696

[Marco Rogers]

I tried this out. It looks like it has less to do with the fact that
you're using buffers and more with the delay while you load the
buffers. I timed it and it takes approx 500ms. That seems to be
enough time for the gc to run. I confirmed this by combining our two
tests. If i run the buffer load and do the xml parse, the memory
stays low.

My guess is that because buffer memory is allocated manually, outside
of v8 space, it's collected and freed correctly. The issue is with
persistent v8 handles because the destructors aren't called in a
timely fashion. I'm thinking the problem is that we need to register
the memory allocated by libmxl for nodes with v8. It seems that v8
doesn't keep track of memory allocated by other libs so it doesn't
count that when it's cosidering when to run the gc. See this article
for a better explanation.

http://create.tpsitulsa.com/wiki/V8/Garbage_Collection

I'mg gonna look into this over the weekend.

:Marco