Re: [nodejs] Node Addon: Threads and libuv ref counter (C++)

[Dean Mao]

There's good slides on doing it here:  http://kkaefer.github.com/node-cpp-modules/

I don't think you need a "proper" way of creating threads -- the only thing you can't do is have the thread access the v8 runtime since js is single-threaded.  You can use uv_queue_work and just have the thread block when you need to.  I've been using Qt and it seems quite easy.  Qt has it's own application object that is usually run separately from node.

Although I would agree about libuv not having the best docs...  you'd essentially read the header file and guess what goes on.

On Fri, Jun 15, 2012 at 6:08 PM, wtfux <wtfu...@googlemail.com> wrote:

Hey guys,

I'm quite a beginner at C++ but I still try to create a node addon in order to use a c++ library in node.

I got it working partially but I have a few problems and questions:

1. The library I'm using will stay during the whole life of the node process and needs to run in an own thread. Is there a "native" way to create threads? I'm using pthreads but I guess there is a better way. I looked into libuv and in the last git master there is a function `uv_thread_create`. Is this what I'm looking for? I'm currently building with latest node.js stable and this function is not yet implemented there. I could probably also use `uv_queue_work` to create a worker in Node's thread pool but I believe this is not a good idea since the worker will never complete and will take a thread from the pool for the whole runtime.

2. I have trouble using libuv's ref counter in order to keep the main Node thread stay alive. Node should stay until the thread from question 1 exists and should be able to handle callbacks in the meantime. (I didn't implement callbacks yet)

In order to do this I called `uv_ref (uv_default_loop ());` and Node will stay open. However here is my problem:

When the thread is about to exit I call `uv_unref(uv_default_loop());` but Node will not terminate. `uv_loop_refcount(uv_default_loop())` shows 1 before that call and 0 after. Shouldn't Node terminate when the count is 0? Is it because I'm calling from a different thread? Or is this even a complete wrong way to keep Node stay open? Without any good documentation or tutorial I find libuv hard to understand.

Thanks for your time!

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[carter-thaxton]

It might be worth mentioning that in v0.7 (unstable), and soon in v0.8 (stable), the signatures of uv_ref() and uv_unref() have changed.

Old:

- uv_ref(uv_loop_t *loop)

- uv_unref(uv_loop_t *loop)

New:

- uv_ref(uv_handle_t *handle)

- uv_unref(uv_handle_t *handle)

That means you can't really use uv_default_loop() anymore.  Instead, you need to pass a handle to something, e.g. uv_async_t.

This is going to break a lot of native modules in the transition to v0.8.  I spent some time yesterday trying to update kkaeffer's node-sqlite3, and ran into issues.

Has anyone else had any luck making this transition, and has anyone written up something describing the changes they had to make?  I'm sure that would be helpful.

[Nathan Rajlich]

The easiest way to deal with the uv_ref/unref() change is to use the NODE_VERSION_AT_LEAST macro from node_version.h. This is what I did for node-ffi: https://github.com/rbranson/node-ffi/blob/02588f775de89aecde82a43bc537dd09e71d958d/src/callback_info.cc#L135-139

[Ben Noordhuis]

On Sat, Jun 16, 2012 at 3:08 AM, wtfux <wtfu...@googlemail.com> wrote:
> Hey guys,
> I'm quite a beginner at C++ but I still try to create a node addon in order
> to use a c++ library in node.
> I got it working partially but I have a few problems and questions:
>
> 1. The library I'm using will stay during the whole life of the node process
> and needs to run in an own thread. Is there a "native" way to create
> threads? I'm using pthreads but I guess there is a better way. I looked into
> libuv and in the last git master there is a function `uv_thread_create`. Is
> this what I'm looking for? I'm currently building with latest node.js stable
> and this function is not yet implemented there. I could probably also use
> `uv_queue_work` to create a worker in Node's thread pool but I believe this
> is not a good idea since the worker will never complete and will take a
> thread from the pool for the whole runtime.

uv_thread_create() is your best option but, as you mention, it's not
available in v0.6.

> 2. I have trouble using libuv's ref counter in order to keep the main Node
> thread stay alive. Node should stay until the thread from question 1 exists
> and should be able to handle callbacks in the meantime. (I didn't implement
> callbacks yet)
> In order to do this I called `uv_ref (uv_default_loop ());` and Node will
> stay open. However here is my problem:
> When the thread is about to exit I call `uv_unref(uv_default_loop());` but
> Node will not terminate. `uv_loop_refcount(uv_default_loop())` shows 1
> before that call and 0 after. Shouldn't Node terminate when the count is 0?
> Is it because I'm calling from a different thread? Or is this even a
> complete wrong way to keep Node stay open? Without any good documentation or
> tutorial I find libuv hard to understand.

Don't do that. libuv is *not* thread-safe with the sole exception of
uv_async_send().

Set up an async handle before you create the worker thread. When the
thread is about to quit, have it wake up the main thread with
uv_async_send(). Unref the loop and close the handle in your callback.
(That's for v0.6. In v0.8, you only need to close the handle.)

I idle in #libuv on freenode.org, feel free to pop in and ask questions.

[wtfux]

Ben Noordhuis wrote:

uv_thread_create() is your best option but, as you mention, it's not
available in v0.6.

 

I switched over to uv_queue_work() and uv_thread_create() (depending on NODE_VERSION_AT_LEAST as mention in this mail thread).

Still not sure about queue_work but I can still change this later.

Don't do that. libuv is *not* thread-safe with the sole exception of
uv_async_send().

Set up an async handle before you create the worker thread. When the
thread is about to quit, have it wake up the main thread with
uv_async_send(). Unref the loop and close the handle in your callback.
(That's for v0.6. In v0.8, you only need to close the handle.)

With uv_queue_work Node will automatically stay open until the thread quits. Do I still need uv_ref or this async_send stuff?

What I need but yet have to implement is that the Node's main thread will stay alive and can handle callbacks just like with any other async addon or like a web server that stays alive and responses to incoming connections.

I idle in #libuv on freenode.org, feel free to pop in and ask questions.

Thanks, I'll join and bug you when I have further questions/problems. ;)

 

[Ben Noordhuis]

On Mon, Jun 18, 2012 at 2:11 AM, wtfux <wtfu...@googlemail.com> wrote:
> With uv_queue_work Node will automatically stay open until the thread quits.
> Do I still need uv_ref or this async_send stuff?

Not for thread management but...

> What I need but yet have to implement is that the Node's main thread will
> stay alive and can handle callbacks just like with any other async addon or
> like a web server that stays alive and responses to incoming connections.

...you will need an async handle for this (and some kind of message
passing between threads; libuv provides cross-platform mutexes,
semaphores, etc.)