capnp::messageToFlatArray to pre-allocated buffer?

[Hui min]

Hi CapnProto Team,

I understand the API to obtain a flat buffer of the built message is capnp::messageToFlatArray.

However, the current API forces you to save the buffer on a returned kj::Array<capnp::word> words object. Is there a way I could pass in a pre-allocated buffer, and let CapnProto to use that for serialisation (which essentially is creating a header + concat all the segments?)?

Currently, some of the API has to be implemented in this way, when pre-allocated buffer is present. This introduced an extra copy and serialisation time.

https://github.com/eclipse-ecal/ecal/blob/master/ecal/core/include/ecal/msg/capnproto/publisher.h#L147-L150

A good kind of API would be like this:
https://github.com/eclipse-ecal/ecal/blob/master/ecal/core/include/ecal/msg/protobuf/publisher.h#L162

Thanks in advance!

[Kenton Varda]

Hi Hui,

Sorry for the slow reply, I was on leave.

messageToFlatArray() is a convenience function for when you don't mind allocation and copying. If you want more control over serialization, you can use MessageBuilder::getSegmentsForOutput() to get direct pointers to the message's underlying buffer space, without copying anything. You can then serialize these segments as you see fit. Note that you are responsible for tracking the segment boundaries in this case. On the receiving end, pass the same array-of-arrays into SegmentArrayMessageReader to parse the message. Again, this doesn't perform any allocation or copies; that's up to you to manage.

Another option is to implement your own kj::OutputStream, and then use capnp::writeMessage() to write a message to the stream. This will always write the entire message using one single call to write() on the underlying stream, passing an array-of-arrays that point into the original message buffer -- again, no copies. You could then copy into your scratch buffer or do whatever else you want. An advantage of this approach (using OutputStream instead of getSegmentsForOutput()) is that writeMessage() will also build a segment table for you and include that in the output, so on the receiving end you can use FlatArrayMessageReader instead of SegmentArrayMessageReader.

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/70bbe8b2-c06d-495f-ac6d-6eedecbdd305n%40googlegroups.com.