Receiving CAPNP Messages over ZeroMQ Multipart Messages with zero-copy?

[Stephan Opfer]

Hi,

I would like to receive capnp messages over zeromq without extra copying the data from zeromq into capnp messages.

My current approach is based on these articles/tutorials/posts:

• ZeroMQ - Multipart + ZeroCopy: http://zeromq.org/blog:zero-copy
• Old post about sending and receiving via ZeroMQ + Capnp: https://groups.google.com/d/msg/capnproto/ClDjGbO7egA/JZjlsUr5kCkJ
• Question about ownership of CAPNP messages, while sending via ZeroMQ: https://lists.zeromq.org/pipermail/zeromq-dev/2017-March/031525.html

So the sending part (untested & uncompiled) is currently like this:

   
    auto segments = message.getSegmentsForOutput();

    auto it = segments.begin();
    auto i = segments.size();
    assert(i != 0);
    while (--i != 0)
    {
        zmq_send_const(this->socket, reinterpret_cast<char const *>(&(*it)[0]), it->size() * sizeof(capnp::word),
                       ZMQ_SNDMORE);
        ++it;
    }
    zmq_send_const(this->socket, reinterpret_cast<char const *>(&(*it)[0]), it->size() * sizeof(capnp::word), 0);

The question is how should I create a capnp message from the received parts of a zeromq multipart message? I read about word alignment, which is taken care of in the second of the above links. But it looks like a lot of copies, could they be avoided?

General hints/advices are welcome too. :)

Greetings,

  Stephan

[Kenton Varda]

Hi Stephan,

You'll want to pass the segments to capnp::SegmentArrayMessageReader.

But, the segments do need to be aligned, and if ZeroMQ doesn't guarantee this, you might be stuck copying. You could of course check dynamically whether the pointer is properly-aligned and skip the copy in that case, but if the alignment is random then you'd only expect to avoid the copy 1 time in 8, which is not so exciting... :/

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/capnproto.

[Stephan Opfer]

Hi Kenton,

I run into the problem, that I need to use UDP and Multi-Part messages are not supported by zeromq while using UDP. The goal of Multi-Part messages are to be delivered in an all or nothing principle. This guarantee is impossible to hold, according to some guys from the ZeroMQ Mailinglist, if you use UDP.

Greetings,
  Stephan

[Kenton Varda]

Hi Stephan,

When using UDP, is there a limit on message size? I would think that if ZeroMQ doesn't allow multi-part messages with UDP, it's because they're trying to fit each message in a single packet, which would imply that your messages have to be less than 64k and should probably be kept under the network MTU which is typically 512-1500 bytes.

If you know your messages will always be small, then you can use the constructor parameters to MallocMessageBuilder to make sure that the first segment is always large enough for the whole message. Then, your message will always be 1 segment and you only need to send that segment.

-Kenton

--