Reading messages from a stream with pycapnp ... how big is a message?

9 views
Skip to first unread message

Andrew Schweitzer

unread,
Nov 3, 2021, 1:45:33 PMNov 3
to capn...@googlegroups.com
Hi capnproto people,

We are transmitting a stream of capnproto messages. The messages are of different types. There is no in-line framing in the stream of bytes. We receive the stream as an array of bytes alongside the list of capnproto schemas to use to decode the stream. I am trying to decode the messages from the stream using pycapnp and I'm having trouble figuring out how large a message is so that I can advance the pointer to the next message.

My code looks like this:

next_buf = body
for message_class in message_classes:
 message = message_class.from_bytes(next_buf)
 next_buf = next_buf[? what goes here ? :]


The message read with from_bytes() is correct, but how many bytes did it consume?

* If I use the read(file) API, the file is correctly advanced, but then I have to write the stream to a file.
* The message has a total_size.word_count member ... but the value of this field appears to be incorrect for purposes of advancing the buffer pointer. As far as I can tell it is returning the total number of words, omitting 1 word for segment count and 1 word / segment.
* I can calculate the number by bytes consumed using to_segments() and then porting c++ code in computeSerializedSizeInWords() ... but am I supposed to do that? to_segments() copies all the data (again) ... so that doesn't seem right.

I feel like I'm missing something obvious. How should I calculate message size and/or read messages from a stream?

Thanks,

Andy
Reply all
Reply to author
Forward
0 new messages