Using V8 object in libeio thread pull function (segfault)
Oleg "Sannis" Efimov
creation from "after" function into function, executed in thread pull
[2]. As I think, I wrap this creation into "HandleScope scope;" and
"scope.Close()", but when I run doSomething2 function with callback in
big loop this sometimes caused segfault. When I run test2_failed.js
with node_g segfauls causes more frequently, so i think this is
because js_result deletes by garbage collector. I'll be glad if
someone could explain to me what went wrong and how to work with the
V8 in the thread pull.
To try this you should clore repo, build addon
$> node-waf configure build
and then run tests:
$> node_g test2_failed.js
[1]: http://github.com/isaacs/node-async-simple
[2]: http://github.com/Sannis/node-async-simple
Ben Noordhuis
Oleg "Sannis" Efimov
There is no hope to correct this situation?
Ben Noordhuis
> There is no hope to correct this situation?
Don't think so unless you want to go the Python GIL route (just say no!).
Oleg "Sannis" Efimov
'after' function in my binary addons.
On Sep 28, 12:12 am, Ben Noordhuis <i...@bnoordhuis.nl> wrote:
Marco Rogers
into the thread? Why can't you do all of your work with C data
structures and just wrap everything back into js afterwards? If you
need to do javascript work in a side thread you should be using child
processes or something.
:Marco
On Sep 27, 4:19 pm, "Oleg \"Sannis\" Efimov" <efimo...@gmail.com>
Oleg "Sannis" Efimov
reduce delays due to data conversion from C + + in the V8 in the case
of large amounts of data from the database. If this occurs in a
separate thread, then it is not wasted the main event loop time.
Orlando Vazquez
statement fetchAll method that returns all rows. The downside to this
approach (not that I have many better ideas...) is that you will
necessarily end up building the data-structures twice. Once, in your
pure C++ in the thread pool, and then again out of the threadpool,
with v8 objects. This will end up occupying memory (possibly a
significant amount) in two places, until the C++ data-structure can be
converted and freed. In my case, it's just a linked list of linked
lists. If you're dealing lots of data this could be a problem.
I have an idea that I would like to test soon: for one C++ function
called from JavaScript to make multiple trips into and out of the
thread-pool (i.e. call eio_custom again from the eio "after"
function). This way, you could maintain a "in progress" data-structure
that you add to every time you come out of the thread pool with your
little "bite sized" data structures. In the node-sqlite case, I'd have
a v8 array, and pull out say, 100 rows at a time to append to it.
Of course, this supposes that you have a job that can be broken up
into pieces (which works great for sqlite.)
Anyways, just something to think about :)
> --
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com.
> To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
>
>
--
Orlando Vazquez
Ben Noordhuis
> I have ran into this while writing node-sqlite, particularly the
> statement fetchAll method that returns all rows. The downside to this
> approach (not that I have many better ideas...) is that you will
> necessarily end up building the data-structures twice. Once, in your
> pure C++ in the thread pool, and then again out of the threadpool,
> with v8 objects. This will end up occupying memory (possibly a
> significant amount) in two places, until the C++ data-structure can be
> converted and freed. In my case, it's just a linked list of linked
> lists. If you're dealing lots of data this could be a problem.
Can't you wrap the C++ class/struct with ObjectWrap::Wrap() and expose
it to V8 that way (all from the main thread, of course)? Saves copying
the data around.
Matthias Ernst
>
> I have ran into this while writing node-sqlite, particularly the
> statement fetchAll method that returns all rows. The downside to this
> approach (not that I have many better ideas...) is that you will
> necessarily end up building the data-structures twice. Once, in your
> pure C++ in the thread pool, and then again out of the threadpool,
> with v8 objects. This will end up occupying memory (possibly a
> significant amount) in two places, until the C++ data-structure can be
> converted and freed. In my case, it's just a linked list of linked
> lists. If you're dealing lots of data this could be a problem.
While it is not an answer to the copying issue, the node protocol
buffer mapping (http://code.google.com/p/protobuf-for-node/) can help
you with the code duplication issue. It allows you to formulate the
native interface in terms of a protocol buffer service, automates the
placement on the eio pool, let's you build up the protocol buffer
response while on the eio thread and automates the marshalling to JS
back on the main thread without ever touching either eio or v8 api.
http://code.google.com/p/protobuf-for-node/source/browse/example/protoservice.cc
shows an example of a getpwent wrapper.
Massaging a streaming response into this would be interesting.
Matthias
Ben Noordhuis
> While it is not an answer to the copying issue, the node protocol
> buffer mapping (http://code.google.com/p/protobuf-for-node/) can help
> you with the code duplication issue. It allows you to formulate the
> native interface in terms of a protocol buffer service, automates the
> placement on the eio pool, let's you build up the protocol buffer
> response while on the eio thread and automates the marshalling to JS
> back on the main thread without ever touching either eio or v8 api.
Interesting technique. Is there a performance trade-off compared to a
one-time copy?