C++ buffered writes in Async client

[igor.ber...@gmail.com]

Hi all,

I've trying to implement a high-performance async server in C++ where throughput matters, but also don't want to keep messages too long in a buffer (say up to 200ms). 

WriteOptions::set_buffer_hint() seems like a perfect candidate to enable high throughput. However whenever a Write method is called on ServerAsyncReaderWriter, then it just gets blocked, because it gets buffered! But then ironically I cannot call another Write method, because GRPC API demands another Write to be only called after another successful Write has completed. How then is supposed to work if async buffered Write does not commit the write right away, but then I cannot call Write method in a batch as well? Am I getting this wrong somehow? I appreciate any help.

void grpc_impl::ServerAsyncReaderWriter< W, R >::Write ( const W &  msg, ::grpc::WriteOptions  options, void *  tag  )

inlineoverridevirtual

Request the writing of msg using WriteOptions options with identifying tag tag.

Only one write may be outstanding at any given time. This means that after calling Write, one must wait to receive tag from the completion queue BEFORE calling Write again. WriteOptions options is used to set the write options of this message

Kind Regards,

 Igor

[yas...@google.com]

Hi,

From what I understand, it seems that you are getting bottlenecked by the network. You are right that gRPC allows only one outstanding write at any given time but that decision itself probably won't affect the throughput much. If the application does not buffer the messages, it would instead need to buffered by gRPC and the end performance would remain about the same.

[Igor Bershadsky]

What is the point of WriteOptions::set_buffer_hint() then? If only one write can be outstanding anyway, then I cannot simply fill-up the socket buffer for WriteOptions::set_buffer_hint()  to work, because the second write can only happen after the first write has been committed and then acked the event queue! This must definitely affect throughput as it is not at-par to just keep pushing data to socket write buffer. I do one write, then wait for the event loop to tells me "ok your write is committed, you can do another one". This is inherently slow by design due to overcommunication for writing messages.

[yas...@google.com]

Admittedly, set_buffer_hint() is not used much, and all it does is to sometimes avoid immediate writes by the HTTP/2 transport in gRPC. Your usecase does not seem one where this behavior is going to be useful. 

To be clear, a write is considered to be "committed" when it passes flow control in gRPC's HTTP/2 layer (not TCP) and it will not always result in bytes being sent out to the socket immediately.