uv_accept in one thread and uv_read_start from other thread

[Tigran Bayburtsyan]

Hi All.

I'm writing multithreaded application based on libuv and now I'm in trouble.

I can't understand how to accept connection from one thread, read some validation data from it and then transfer that connection to another thread for reading remaining data.

I found some examples for multiprocess communication using UV_PIPE-s, but in all that examples master process just sending notification to accept connection.

What I need is accept connection from one thread and then without closing connection start reading from another thread.

Is there any solution for that ? even if I need to hack something.

// Accept connection
    if(uv_accept(server, (uv_stream_t *)client) == 0) {

       ..................
       ..................

        // how can I call uv_read_start for a different loop ???
        int r = uv_read_start((uv_stream_t *)client, alloc_cb, on_read);
        if(r) {
            uv_close((uv_handle_t*) client, close_client);
            return;
        }

    } else {
        uv_close((uv_handle_t*) client, close_client);
    }

  

[Saúl Ibarra Corretgé]

You need to send the handle from one loop to another, just in the
multi-process examples you saw. A handle it tied to a given loop, so if
you want to use from a different one, you must use uv_write2 and send
the handle over a pipe to the loop running on the other thread.


Cheers,

--
Saúl Ibarra Corretgé
http://bettercallsaghul.com

[Tigran Bayburtsyan]

Thanks for replay, almost same thing is written in documentation and couple of multiprocess examples.

Can you share with me super super super super simple example of code ? 
Thanks

[Saúl Ibarra Corretgé]

On 26/08/16 15:13, Tigran Bayburtsyan wrote:
> Thanks for replay, almost same thing is written in documentation and
> couple of multiprocess examples.
> Can you share with me super super super super simple example of code ?
> Thanks
>

I don't have an example handy, but here is a test which does exactly
that:
https://github.com/libuv/libuv/blob/v1.x/test/test-ipc-send-recv.c#L265

[Tigran Bayburtsyan]

It's sad, I tried to get example out of that file few days ago, but it's not related to what I'm trying to do.

Anyway thanks, will try to hack something :)

[Tigran Bayburtsyan]

Just wanted to give some update.

It looks strange but I've just changed loop pointer in socket and it started working from another thread.

I'm doing in this way

1. Running 4 loops for reading data and 1 loop for accepting connection
2. Accepting connection, authenticating, stopping read process and changing loop pointer to one of the 4 other loops
3. Starting read process

I've surprised but it works very well, even on data load.

Am I doing wrong, by changing loop pointer manually ?

[Saúl Ibarra Corretgé]

On 08/26/2016 10:34 PM, Tigran Bayburtsyan wrote:
> Just wanted to give some update.
> It looks strange but I've just changed loop pointer in socket and it
> started working from another thread.
>
> I'm doing in this way
>

> 1. Running 4 loops for reading data and 1 loop for accepting connection
> 2. Accepting connection, authenticating, stopping read process and

> changing loop pointer to one of the 4 other loops

> 3. Starting read process

>
> I've surprised but it works very well, even on data load.
> Am I doing wrong, by changing loop pointer manually ?
>

That's a terrible idea. You shouldn't change the loop pointer of any
handle, it's considered "readonly"
http://docs.libuv.org/en/v1.x/handle.html#c.uv_handle_t.loop Now, this
is C so you're allowed to shoot yourself in the foot. I can't prevent
you from doing it, but note that when you application blows up you
voided the warranty :-)

As I mentioned earlier: use uv_write2, that's why it exists, to send a
handle over a pipe. All you need is to have 4 pipes connecting the
"main" loop with each of the workers, and send the accepted connection
(or read a bit, send it later, it doesn't matter) to any of the workers.

Cheers,

--
Saúl Ibarra Corretgé

bettercallsaghul.com

[Yuriy Yevtukhov]

Hi.

 

I have a similar task - process incoming connections in different thread. And I don't have fixed set of working threads, they are started on demand and depending on settings which can, in particular, say to start new thread for each incoming connection.

 

Creation and processing of pipe just to transfer socket inside one process is not trivial task. Sorry, but it looks rather ... over-engineered, to use this IPC method for inter-thread communication when without libuv the task is very simple - just pass accepted FD to new thread. Especially when there are many independent listening sockets (each can listen in different loop/thread). And also using a pipe makes accepting new connection rather long-timed task because requires several loop iterations.

 My idea is to accept connections in one thread / loop (same as listening) and then dup() accepted socket, close original accepted socket, pass dup'ed socket to new thread / loop. On Unix this looks like:

uv_tcp_t *client_sock=new uv_tcp_t;  //must not be local to be correctly closed

uv_tcp_init(server->loop, client_sock);

uv_accept((uv_stream_t*)&server->sock, (uv_stream_t*)client_sock);

uv_os_fd_t client_fd, client_fd_dup;
uv_fileno((uv_handle_t*)client_sock, &client_fd);

client_fd_dup=dup(client_fd);

uv_close((uv_handle_t*)&client_sock,  [](uv_handle_t* handle) -> void {
  uv_tcp_t *client_sock=(uv_tcp_t *)handle;
  delete  client_sock;
});
//pass client_fd_dup to another thread immediately OR inside uv_close callback
//and do "uv_tcp_open(&client->sock, client_fd_dup)" for client->sock which was inited for loop of that thread

This method seams to work and be legal for libuv. Any problems to expect?