Multithreading RPC servers

[eamonn...@demilletech.net]

Hi,

Maybe this is an easy question, maybe it's not. I guess we're about to find out together. I'm working on application that I need to be able to handle large amounts of requests in parallel. It needs to be fast in user time, not CPU time. If I hit this with as many requests per second as it can take, it begins to get bottlenecked by single-threaded performance.

My question is: is there space in the capnproto protocol to have multithreaded receivers, or have the scheduling use multiple threads for its event queue? I've seen it talked about, but unfortunately, I've not found any documentation regarding it. I've been using go-capnproto2 for all of this, but my reading has suggested I'm plainly missing something.

On another side, mentally at least, I'd imagine that it would be nice to allocate a number of worker threads to handle queueing, but that may be a fight (against myself) for another day. Or, allowing me to allocate tasks to another thread.

Thanks in advance! And sorry for the kinda dumb question.

--Eamonn

[Ian Denhardt]

Quoting eamonn.nugent via Cap'n Proto (2019-05-23 20:50:04)

> My question is: is there space in the capnproto protocol to have
> multithreaded receivers, or have the scheduling use multiple threads
> for its event queue?

By receivers do you mean a single object?

If so, the default is for messages to be serviced in-order. In the Go
implementation you can unblock future calls before returning by calling
server.Ack():

https://godoc.org/zombiezen.com/go/capnproto2/server#Ack

Hope that helps.

-Ian

[Kenton Varda]

Hi Eamonn,

Definitely not a dumb question! This kind of thing is the topic of a great deal of argument and differing opinions from smart people. :)

I don't know much about how the Go implementation works -- I imagine due to the nature of Go, it's more likely to utilize multiple threads. As Ian points out, by default a capability server in the Go implementation will only accept one method call at a time, but you can call server.Ack() at the start of the method to "unblock" the next call to happen concurrently.

In Cap'n Proto's C++, I made the design decision to focus on single-threaded event loops, because the synchronization involved in allowing multiple threads access the same objects is not just difficult to get right, but also expensive. In C++, the right way to build a multi-threaded server is to create a separate event loop in each thread. Any particular RPC connection -- and all the calls and objects associated with it -- will have to be owned by one particular thread and handled there, but different connections could potentially be distributed across different threads.

In the future, I'd like to extend KJ so that event loops in separate threads can signal each other more easily. (Currently, you more-or-less have to create a socketpair to communicate between threads.) I'd also like to implement Cap'n Proto's 3-party handoff protocol. Perhaps this could allow threads to seamlessly "hand off" to objects in other threads, with the caveat that the client will need to form a new connection to the new thread.

I don't expect the C++ implementation will ever allow multiple threads to share a single event loop or handle events from a single connection. The synchronization complexity is just too high for that.

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/4405b081-d9d7-49aa-a128-d0c168daf4fc%40googlegroups.com.

[Ian Denhardt]

Quoting 'Kenton Varda' via Cap'n Proto (2019-05-24 19:39:38)

> I don't know much about how the Go implementation works -- I imagine
> due to the nature of Go, it's more likely to utilize multiple threads.

Indeed, with the Go implementation you should generally get as much CPU
parallelism as E-order permits.

-Ian