Re: [nodejs] [libuv] thread, uv_loop_new and uv_accept dangerous cocktail

[Ben Noordhuis]

On Sun, Jul 1, 2012 at 7:32 AM, Mathieu Garaud <mathieu...@gmail.com> wrote:
> Hello,
>
> I'm playing with the sample code uv_webserver released by Ryan Dahl and I
> want to add thread support for incoming connection.
>
> To make it short I'm just trying to thread the on_connect after uv_listen to
> make sure that all the tcp connections will be distributed over all the
> cpus available on my computer.
>
> After my naive attempt of creating a thread and new loop and try to use
> uv_accept I found those lines of code in unix/stream.c (line: 218-219):
>
> /* TODO document this */
> assert(server->loop == client->loop);
>
> I read the src code and I understand the dependency (mainly around loop)
> between the server and the client so I wanted to know if there is
> a workaround ?

No.

libuv is not concurrency safe. You cannot access a uv_loop_t or
anything that's associated with it (uv_req_t, uv_handle_t, etc.)
simultaneously from multiple threads.

libuv is not thread safe either. Even with proper locking in place,
using libuv structures from multiple threads is not safe. That's
mostly a uv-win limitation but uv-unix doesn't make any guarantees
either.

> BTW I thought that maybe I could use memcpy on uv_stream_t and assign the
> new loop to server variable, but I have no idea if the struct will be
> correctly copied and if it won't introduce more problems. Do you know if
> there is a method to copy and move uv_stream_t between loops properly?

Not at the moment. We used to support it when we were working on
isolates support in node. When isolates got dropped, we dropped that
feature from libuv as well.

If you have a strong need for such a feature, open an issue at
https://github.com/joyent/libuv/issues (and please explain your use
case.)

[Mathieu Garaud]

Hello Ben,

Thanks for your quick reply.

I understand that libuv is not thread safe and it's made by design. I'd like to know if we could benefit from the best of both world: multi cores cpus + event based loop without the slow downs and the complexity of mutexes, semaphores ...

Correct me if I'm wrong but the problem of passing stream from one loop to another means implementing a thread safe mechanism, isn't it?

Is there is any other way to implement it? I though that if I could mark a stream (or put in stasis but it seems harder to implement because I had to sync the latent data) then the other loop could import it + register file descriptor with a minimun lock risk or time consumed. This implementation will introduce a limitation one stream will only be able to be attached to 1 loop and it will be async.

Anyway, you probably already think of it trying to implement isolates support for node.

Thanks in advance for your help.

Cheers,

Mathieu

[Ben Noordhuis]

On Mon, Jul 2, 2012 at 1:39 AM, Mathieu Garaud <mathieu...@gmail.com> wrote:
> Hello Ben,
>
> Thanks for your quick reply.
>
> I understand that libuv is not thread safe and it's made by design. I'd like
> to know if we could benefit from the best of both world: multi cores cpus +
> event based loop without the slow downs and the complexity of mutexes,
> semaphores ...
>
> Correct me if I'm wrong but the problem of passing stream from one loop to
> another means implementing a thread safe mechanism, isn't it?
>
> Is there is any other way to implement it? I though that if I could mark a
> stream (or put in stasis but it seems harder to implement because I had to
> sync the latent data) then the other loop could import it + register file
> descriptor with a minimun lock risk or time consumed. This implementation
> will introduce a limitation one stream will only be able to be attached to 1
> loop and it will be async.
>
> Anyway, you probably already think of it trying to implement isolates
> support for node.

I'm afraid there's no convenient support for in-process sharing of
handles at the moment. What can you do is the following (I'm assuming
you have some kind of TCP server that you want to share across
threads):

1. (thread A) create your uv_tcp_t server handle (i.e. bind and listen)
2. (thread A) create a uv_pipe_t and bind it to a path.
3. (thread B) connect to the pipe
4. (thread A) send over the server handle with uv_write2()
5. (thread B) receive handle and start listening

In case you're interested, [1] is the commit that removed uv_import()
and uv_export(). In hindsight, it may have been better to leave them
in.

Then again, the approach outlined above works and has the benefit that
it's intrinsically thread-safe - all the hard work is done by the
operating system.

[1] https://github.com/joyent/libuv/commit/c5aa86b

[Mathieu Garaud]

Hi,

Thanks for the tip! I'll implement this solution this evening and I'll see if I'm able to increase the number of request/s of this HTTP server can handle.

Cheers,

Mathieu

[Shripad K]

Ngninx/Stud fork()s and the child runs the event loop; the parent just does a bind(). Since stud uses libev you can also look at the implementation in stud:

https://github.com/bumptech/stud/blob/master/stud.c#L1822

https://github.com/bumptech/stud/blob/master/stud.c#L1853

https://github.com/bumptech/stud/blob/master/stud.c#L1613

https://github.com/bumptech/stud/blob/master/stud.c#L1515

On Wed, Aug 29, 2012 at 2:45 PM, Zhang Hu <iamzh...@gmail.com> wrote:

Hi Mathieu,

I'm thinking the same problem as you had. If I'm right, nginx's work-thread machnism can benifit the multiple CPUs/cores. Nginx listens one sockets by multiple processes.

How is your solution doing?

Regards,
Tiger

Does this work well if there are more threads? Will the threads not contest with each other for performing an accept() if there is an event loop per thread listening for incoming connections? Or am I missing something?

 

In case you're interested, [1] is the commit that removed uv_import()
and uv_export(). In hindsight, it may have been better to leave them
in.

Then again, the approach outlined above works and has the benefit that
it's intrinsically thread-safe - all the hard work is done by the
operating system.

[1] https://github.com/joyent/libuv/commit/c5aa86b

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Ben Noordhuis]

On Wed, Aug 29, 2012 at 1:36 PM, Shripad K <assortme...@gmail.com> wrote:
> Ngninx/Stud fork()s and the child runs the event loop; the parent just does
> a bind(). Since stud uses libev you can also look at the implementation in
> stud:
>
> https://github.com/bumptech/stud/blob/master/stud.c#L1822
> https://github.com/bumptech/stud/blob/master/stud.c#L1853
> https://github.com/bumptech/stud/blob/master/stud.c#L1613
> https://github.com/bumptech/stud/blob/master/stud.c#L1515

That won't work with libuv out of the box, it marks all file
descriptors as close-on-exec to avoid leaking file descriptors to
child processes.

A quick workaround is to mark the file descriptor as inheritable again
with fcntl(stream->fd, F_SETFD, 0). (But note that fd is a private
field and may be renamed or removed in the future.)

[Mathieu Garaud]

Hi Zhang,

I implemented Ben's solution but I faced exactly the same issue. I mean even if I pass uv_stream_t over the pipe using uv_write2 the stream is still attached to the wrong event loop. Hence the callbacks are executed on the wrong thread.

However I checked the code at the revision Ben's referred to in one of his email and I think that if you understand how libev works you can unregister all the event listeners and move them to the new event loop on another thread. I'm sorry I didn't go further in my investigation.

However reading stream.c again I don't remember why the streamServer and streamClient has to be on the same event loop.

Ben knows probably the reason but if this assertion wasn't there I'm pretty sure that we could implement the threading much mor easier.

I'm so sorry but I didn't push further my investigations.

Regards,

Mathieu

On Wednesday, August 29, 2012 11:15:55 AM UTC+2, Zhang Hu wrote:

Hi Mathieu,

I'm thinking the same problem as you had. If I'm right, nginx's work-thread machnism can benifit the multiple CPUs/cores. Nginx listens one sockets by multiple processes.

How is your solution doing?

Regards,
Tiger

On Monday, July 2, 2012 4:28:13 PM UTC+8, Mathieu Garaud wrote:

[Shripad K]

Hi Mathieu,

I tried Ben's solution in a side-project of mine (which is written in D and inspired by NodeJS). It works but is much slower than the single threaded event loop approach. So even if you get multi-threaded eventloop there is latency added. Also, you need to make some tweaks. Eg: in libev, setting ev_set_io_collect_interval to a higher value so that not all pending events are handled by 1 thread (which is going to take you back to single threaded approach with other threads freely spinning), but instead is handled by as many threads as is available. In libev at least there is no way to specify how many events should be handled in a single iteration (at least that I know of and I guess its the same with libuv too). I think multi-threaded event loop is more cumbersome than just fork() and loop in child process (with Ben's suggestion about making the descriptor inheritable for libuv).

Shripad K

--

[Zhang Hu]

Hi guys,

Thanks for your swift replies.
I checked the source codes of Stud which Shripad K mentioned. The design of Stud (simillar with Nginx) is what I want for our TCP frontend service.
Parent process binds/listens a socket, child processes run event loops. Then we can benifit event loop and multiple CPUs/cores.
I developed an echo service based on the source codes of Stud. It works well and fast. So I swithed to libev.

Thanks,
Tiger