Exceptions when reading protobuf messages in Java which were created in C++

[platzhirsch]

I am using **protobuf** now for some weeks, but I still keep getting
exceptions when parsing protobuf messages in **Java**.

I use **C++** to create my protobuf messages and send them with
**boost sockets** to a server socket where the Java client ist
listening. The C++ code for transmitting the message is this:

boost::asio::streambuf b;
std::ostream os(&b);

ZeroCopyOutputStream *raw_output = new OstreamOutputStream(&os);
CodedOutputStream *coded_output = new CodedOutputStream(raw_output);

coded_output->WriteVarint32(agentMessage.ByteSize());
agentMessage.SerializeToCodedStream(coded_output);

delete coded_output;
delete raw_output;

boost::system::error_code ignored_error;

boost::asio::async_write(socket, b.data(), boost::bind(
&MessageService::handle_write, this,
boost::asio::placeholders::error));


As you can see I write with `WriteVarint32` the length of the message,
thus the Java side should know by using `parseDelimitedFrom` how far
it should read:

AgentMessage agentMessage = AgentMessageProtos.AgentMessage
.parseDelimitedFrom(socket.getInputStream());

But it's no help, I keep getting these kind of Exceptions:

Protocol message contained an invalid tag (zero).
Message missing required fields: ...

It is **important** to know, that these exceptions are not thrown on
every message. This is only a fraction of the messages I receive the
most work out just fine - still I would like to fix this since I do
not want to omit the messages.

I would be really gratful if someone could help me out or spent his
ideas.

[platzhirsch]

Some other exceptions I receive as well:

Protocol message tag had invalid wire type.
Protocol message end-group tag did not match expected tag.
While parsing a protocol message, the input ended unexpectedly in the
middle of a field. This could mean either than the input has been
truncated or that an embedded message misreported its own length.

[Jason Hsueh]

I'm not familiar with the boost libraries you're using, but the use of "async_write" and the stack-allocated streambuf looks suspect. If nothing jumps out there, I would first check that the data read from the socket in Java exactly matches the data on the C++ side.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To post to this group, send email to prot...@googlegroups.com.
To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.

[platzhirsch]

I find it difficult to check this by so many messages. This approach
was suggested by some people already, I don't know how I would realize
this.

On Jul 6, 6:56 pm, Jason Hsueh <jas...@google.com> wrote:
> I'm not familiar with the boost libraries you're using, but the use of
> "async_write" and the stack-allocated streambuf looks suspect. If nothing
> jumps out there, I would first check that the data read from the socket in
> Java exactly matches the data on the C++ side.
>

[Christopher Smith]

Basic rule: if the memory is used outside the lifetime of the function call, you don't want it on the stack. Async_write very much requires the memory to be around later.

--Chris

[Jason Hsueh]

Can you not control the number of messages?

In any event, I think the problem is the memory lifetime of the streambuf. From the docs on async_write:

b

A basic_streambuf object from which data will be written. Ownership of the streambuf is retained by the caller, which must guarantee that it remains valid until the handler is called.