Read and write from/to a single TCP socket in different event loops

[STU T]

Sorry if this question has been asked before, I've googled a fair bit and haven't been able to find anything about this.

I'm experimenting with porting a library from ASIO to libuv. I have everything working, and libuv seems pretty nice so far, but there's one crucial problem: this is a streaming rather than request/response protocol. ASIO (and plain system sockets) allows handling the read and write direction of a single TCP socket from different threads. As far as I can see, libuv ties a TCP socket to single loop, and the loop is not thread-safe. Is there any way to split the read and write ends of a socket so I can deal with the read and write directions on two different loops?

In case you're wondering about the use-case and why I believe this is necessary: the library uses TCP internally, but allows the user to send and receive data to/from the TCP socket via blocking-style API calls which necessarily occur on a separate user-controlled thread, and therefore can't be integrated directly into libuv's "everything is async" model. The library uses a pair of bounded blocking queues to store outgoing data to be written to the network, and incoming data read from the network, respectively. With libuv, both queues filling up at the same time can cause a deadlock: both the loop and the user thread are stuck waiting for the inbound/outbound queues (respectively) to have free space, and the loop never takes anything off the outbound queue to write to the socket. Fixing this deadlock scenario with one event loop would require some pretty awkward/obtuse code to detects full queues, temporarily store data somewhere else, and call uv_read_stop to prevent any more data coming in until the blockage goes away.

[Saúl Ibarra Corretgé]

Hi!

On 12/12/2017 15:46, STU T wrote:
> Sorry if this question has been asked before, I've googled a fair bit
> and haven't been able to find anything about this.
>
> I'm experimenting with porting a library from ASIO to libuv. I have
> everything working, and libuv seems pretty nice so far, but there's one
> crucial problem: this is a streaming rather than request/response
> protocol. ASIO (and plain system sockets) allows handling the read and
> write direction of a single TCP socket from different threads. As far as
> I can see, libuv ties a TCP socket to single loop, and the loop is not
> thread-safe. Is there any way to split the read and write ends of a
> socket so I can deal with the read and write directions on two different
> loops?
>

Not really. You can alway use async handles to try to start an operation
from a different thread and "make it" thread-safe, but the overhead cost
is probably not going to be worth it.

> In case you're wondering about the use-case and why I believe this is
> necessary: the library uses TCP internally, but allows the user to send
> and receive data to/from the TCP socket via blocking-style API calls
> which necessarily occur on a separate user-controlled thread, and
> therefore can't be integrated directly into libuv's "everything is
> async" model. The library uses a pair of bounded blocking queues to
> store outgoing data to be written to the network, and incoming data read
> from the network, respectively. With libuv, both queues filling up at
> the same time can cause a deadlock: both the loop and the user thread
> are stuck waiting for the inbound/outbound queues (respectively) to have
> free space, and the loop never takes anything off the outbound queue to
> write to the socket. Fixing this deadlock scenario with one event loop
> would require some pretty awkward/obtuse code to detects full queues,
> temporarily store data somewhere else, and call uv_read_stop to prevent
> any more data coming in until the blockage goes away.
>

Yep, I see no other solution right off the bat, I'm afraid.

> --
> You received this message because you are subscribed to the Google
> Groups "libuv" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to libuv+un...@googlegroups.com
> <mailto:libuv+un...@googlegroups.com>.
> To post to this group, send email to li...@googlegroups.com
> <mailto:li...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/libuv.
> For more options, visit https://groups.google.com/d/optout.


--
Saúl Ibarra Corretgé
http://bettercallsaghul.com