How to implement streaming from server to client?

[Dmitriy Lekomtsev]

I've not good in C++ metaprogramming and it is not obvious how to implement streaming from server to client.

I want to use Publish/Subscribe pattern where client (which is behind NAT) connects to server using capnp::EzRpcClient and provide server callback via which server should provide current sensors data.

I've read announcement about  0.8 version where first interface is what I want:

interface MyInterface { 

  streamingCall @0 (callback :Callback) -> (); 

  interface Callback { 

    sendChunk @0 (chunk :Data) -> (); 

  } 

}

For further explanation let's rewrite it to this:

interface SensorsService {

  subscribeToSensors @0 (sensorsIds :List(Int32), callback :ClientCallback) -> ();

  interface ClientCallback {

     sendSensorsDataChuck @0 (sensorId: Int32, chuck :Data) -> ();

  }

}

I've questions about how to implement this on both sides.

Before I've used Cap'n Proto with capnp::EzRpcClient and capnp::EzRpcServ without streaming. I've placed server within Qt thread and communicate with rest of system using standard threading synchronization primitives.

But about streaming I don't understand following.

On client side:

1. How to provide ClientCallback within subscribeToSensors call?

Using creating kj::heap<ClientCallbackImpl> (descendant of ClientCallback::Server)? If so who will owns that kj::heap pointer and how long will it life? 

2. Who will  and how control  lifecycle of that ClientCallback "server"?

3. How to specify to publishing server intention to stop utilizing that ClientCallback instance?

Via additional method (like stop) or "just" destroy instance on client side? If latter then how exactly do this?

On server side:

1. How to send infinite chucks after receiving subscribe request?

In this question: C++ Bidirectional Stream Back to Client example is not  complete. It "describe" (if that even could be called describe) only client-to-server streaming. I've needed to send infinite chucks from server to client (from Qt thread other than event loop of EzRpcServer).

2. Is server able to detect that client (or/and last client) who previously provided callback disconnected and that server should not try to send data anymore?

3. Since in EzRpcServer thread server running blocking forever loop. How to notify server to send data to clients within its thread? I've read in your docs that you do not like to use calling your methods from different threads. Currently I'm using following blocking server start (_threadContactStream used from Qt thread to stop server):

auto myPipe = server.getIoProvider().newOneWayPipe();
_threadContactStream = std::move(myPipe.out);
uint8_t buf = 0;
myPipe.in->read(&buf, 1).wait(server.getWaitScope());

Do I need to use something similar to notify server? But in this case server stopped. So do I need use same pipe to wait again?

PLEASE DO NOT BE LIKE STUPID AMERICANS. Do not answer for any question with "Yes". US citizens likes answer for complex question with non related answer.

Please be as more verbose as you can. Not many people creating own libraries each weekend. So short answers without code is not answer at all.

Thanks in advance for patience.

[Kenton Varda]

Hi Dmitriy,

I had a long set of answers typed up for you, but then I came to your last paragraph:

 

PLEASE DO NOT BE LIKE STUPID AMERICANS. Do not answer for any question with "Yes". US citizens likes answer for complex question with non related answer.

Please be as more verbose as you can. Not many people creating own libraries each weekend. So short answers without code is not answer at all.

Thanks in advance for patience.

Based on this paragraph, I am not interested in answering your questions. Please do not use Cap'n Proto. Maybe you should try gRPC instead.

Thanks,

-Kenton 

[Dmitriy Lekomtsev]

Based on this paragraph, I am not interested in answering your questions. Please do not use Cap'n Proto. Maybe you should try gRPC instead.

It is sad. Your library more useful for Windows than gRPC.

Thanks for your answer anyway.