Re: [capnproto] Re: Use with TCP socket, C++ -> pycapnp, no RPC

[Kenton Varda]

Hi Robert,

> message has too many segments error.

This basically means the data is corrupted somehow. You'll need to track down why the bytes you're receiving are not the same as the bytes you are sending. To debug, you might try reading the bytes into a regular byte buffer and logging them, and also log on the sending end, and see what's different. Then narrow down the issue.

> kj::OutputStream subclass that sends the data via ASIO.

This sounds tricky, since kj::OutputStream does synchronous I/O but ASIO does async. How are you handling the case where the write can't complete immediately?

-Kenton

On Thu, Sep 10, 2020 at 9:51 AM Robert Olivier <robertol...@gmail.com> wrote:

I should have been more clear about how I'm reading the stream on the python server side .....

I consume the 4 byte length header before calling message_capnp.Message.read(socket) as I cannot see how to read the payload directly and then offer the bytes to capnp in lieu of having capnp read directly from the socket. So those header bytes are not accidentally being read by message_capnp.Message.read(socket).

On Thursday, September 10, 2020 at 10:26:24 AM UTC-4 Robert Olivier wrote:

I've scoured the net for a solid example of using capnp purely as a data interchange format sans RPC, streaming over TCP and dont see it. I like the design concept of capnp vs competitors and I want to use it in place of ASN.1 generated code for a project.

However, the documentation is just so scattered and unclear that I can't get a basic test structure sucessfully sent to a python process.

I've implemented my own simple framing, consisting of a 4 byte length that c++ sends to python before the message. I compute the payload length as computeSerializedSizeInWords() * sizeof(capnp::word) and then send it, using an kj::OutputStream subclass that sends the data via ASIO.

My python process reads the length and payload, but when I try to create the message with

message_capnp.Message.read(socket) I get a message has too many segments error.

I have read that vague bit in the Encoding | Serialization Over a Stream section of the home page docs and I cannot tell if that is saying I need to implement that on both sides of the wire, or if capnp::writeMessage() implements that.

Is there some documentation that I'm missing?  Seems strange that such a common use case would be effectively undocumented!  Is use as a pure data-interchange format discouraged?

rjo

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/3fe1dde2-5e30-496b-9469-ab00b946113en%40googlegroups.com.