How to create kj::ArrayPtr<capnp::word> from char *?

[zosia...@gmail.com]

I have raw buffer read from socket and I want to pass/add it to kj::ArrayPtr<capnp::word>. Is there a simple way to do this (without additional copies)?

I've tried answer to this (not the copying one) : https://groups.google.com/forum/#!topic/capnproto/CIUxq-Y4128, with simple code :

    std::vector<unsigned char> bytes(1024);
    kj::ArrayPtr<capnp::word> words(reinterpret_cast<const capnp::byte*>(bytes.data()), bytes.size() / sizeof(capnp::word));

but it gives me

error: no matching function for call to ‘kj::ArrayPtr<capnp::word>::ArrayPtr(const byte*, std::vector<unsigned char>::size_type)’

and I'm kind of stuck.

[Kenton Varda]

Hi zosiasmail,

On Mon, Dec 11, 2017 at 8:43 AM, <zosia...@gmail.com> wrote:

    std::vector<unsigned char> bytes(1024);
    kj::ArrayPtr<capnp::word> words(reinterpret_cast<const capnp::byte*>(bytes.data()), bytes.size() / sizeof(capnp::word));

This code will work if you change the reinterpret_cast to:

    reinterpret_cast<capnp::word*>(bytes.data())

That is, you are casting to the wrong type (and wrong constness).

However, there's a deeper problem this doesn't solve, which is ensuring that your buffer is aligned. There's no guarantee that the bytes in a vector are aligned to a word boundary. Since Cap'n Proto doesn't have a separate decoding step, it's necessary that the message be properly-aligned for direct access of types up to 64 bits.

The trick is to allocate your backing buffer as words in the first place:

    std::vector<capnp::word> words(128);

Now you can read into this vector like:

    read(fd, words.begin(), words.size() * sizeof(capnp::word))

Better yet, don't use std::vector; use kj::Array all the way through:

    auto words = kj::heapArray<capnp::word>(128);

    ssize_t n = read(fd, words.begin(), words.size() * sizeof(capnp::word));

    // TODO: check errors, etc.

    auto messageWords = words.slice(0, n / sizeof(capnp::word));

If you are using an I/O library that insists on giving you strictly bytes with no alignment guarantee, then you might have a problem. You may be forced to make a copy in this case.

-Kenton

[Zosia A]

Thank you for fast reply!

Yeah, I was doing it just how you've shown, but now I'm trying to integrate it with existing project and trying to do this with as little changes possible. I wasn't aware, that vector doesn't guarantee alignment for any standard type, so I will have to stick to the first version. Thank you once again :).

[Kenton Varda]

FWIW, it's actually pretty likely that a vector<char>'s buffer will actually be aligned in practice, because it is almost certainly allocated using malloc() which always returns aligned buffers. But, there's technically no guarantee.

Given this, perhaps you could write some code which *checks* for alignment, and if so, does a reinterpret_cast, but if the buffer isn't aligned, then it falls back to a copy.

    bool aligned = reinterpret_cast<uintptr_t>(bytes.begin()) % sizeof(void*) == 0;

(I use sizeof(void*) as the denominator because 32-bit systems usually require only 32-bit alignment even for 64-bit data types.)

-Kenton