Can't deserialize messageToFlatArray

[Vitali Lovich]

I'm having a weird problem that I'm suspecting comes from invalid usage of the API?

Basically I offload the serialized message (obtained via capnp::messageToFlatArray) to a background thread for appending to a file. However, I'm noticing that the messages are corrupt. I'm wondering if I'm misusing the API in some obvious way? AFAICT writing the output of messageToFlatArray should be identical to sending it to an OutputStream...

Hopefully it's not a glaring oversight on my part like the past couple of messages.

Pseudo-code below:

void append(capnp::MessageBuilder& builder) {

 auto serializedInWords = capnp::messageToFlatArray(builder);

 {

    std::lock_guard lg(mutex_);

    queue_.emplace_back(kj::mv(serializedInWords));

 }

 available_.notify_all();

}

void flushThread() {

  std::vector<kj::Array<const capnp::word>> toFlush;

  // output_ is kj::AutoCloseFd - the thread RAII joins on destruction & is declared after output_ so the I/O should outlive  

  // also the problematic message is one that I write once on start & the I/O gets terminated much later so I don't think it's a race for the I/O.

  kj::FdOutputStream output(output_.get());

  do {

    {

      std::unique_lock lck(available_);

      available_.wait(lck, [&] { return terminated_ || !queue_.empty());

      toFlush = std::move(queue_);
    }

    for (const auto& message : toFlush) {
      auto messageBytes = message.asBytes();

      output.write(messageBytes.begin(), messageBytes.size());
    }        

  } while (!terminated_);

}

[Vitali Lovich]

Additional context I forgot to include. I'm getting identical output with just a naive capnp::writeMessageToFd on the original builder. In the sample I'm looking at I only ever write 1 message. The conversion using the capnp decode tool fails with:

*** ERROR CONVERTING PREVIOUS MESSAGE ***
The following error occurred while converting the message above.
This probably means the input data is invalid/corrupted.
Exception description: expected ref->kind() == WirePointer::STRUCT; Message contains non-struct pointer where struct pointer was expected.

Code location: capnp/layout.c++:2168
*** END ERROR ***

[Vitali Lovich]

And I am an idiot. The message I'm constructing in the builder isn't what I'm serializing it as. I'm sorry for the noise.