Cannot deserialize protobuf data from C++ in Java

[platzhirsch]

My problem is to serialize protobuf data in C++ and deserialize the
data in Java probably. In fact I get **InvalidProtobufferException**
in Java.

Here is my minimal example:

My protobuf file looks like this:

option java_outer_classname = "NameProtos";

message Name {

required string name = 1;
}

Which I compile with:

protoc -I=. --cpp_out=../cpp/ name.proto

and

protoc -I=. --java_out=../java/ name.proto


In C++ I create the name object this way:

Name name;
name.set_name("platzhirsch");
socket.send(name.SerializeAsString);

In Java I read from the socket, until the socket is closed
(socket.send closes the connection, after it entirely wrote the string
passed, so I guess here is no when stop reading issue, isn't there?).

In Java I make the following call in order to deserialize:

`NameProtos.Name name =
NameProtos.Name.parseFrom(ByteString.copyFromUtf8(received))`;

However I always get **InvalidProtocolBufferException**

I have no idea, the received strings are not empty though, they look
like this:
[SPACE]platzhirsch[A Character which cannot be displayed probably]

These character which I cannot read, I also get, when using other
fields like int32. I guess there are just encoded into the string.

[Jason Hsueh]

On Fri, Apr 15, 2011 at 2:06 AM, platzhirsch <konrad...@googlemail.com> wrote:

My problem is to serialize protobuf data in C++ and deserialize the
data in Java probably. In fact I get **InvalidProtobufferException**
in Java.

Here is my minimal example:

My protobuf file looks like this:

   option java_outer_classname = "NameProtos";

   message Name {

      required string name = 1;
   }

Which I compile with:

   protoc -I=. --cpp_out=../cpp/ name.proto

 and

   protoc -I=. --java_out=../java/ name.proto


In C++ I create the name object this way:

   Name name;
   name.set_name("platzhirsch");
   socket.send(name.SerializeAsString);

In Java I read from the socket, until the socket is closed
(socket.send closes the connection, after it entirely wrote the string
passed, so I guess here is no when stop reading issue, isn't there?).

In Java I make the following call in order to deserialize:

`NameProtos.Name name =
NameProtos.Name.parseFrom(ByteString.copyFromUtf8(received))`;

The serialized data is binary and not valid UTF8. The conversion here corrupts the data - make sure to parse the exact bytes that are serialized.

 

However I always get **InvalidProtocolBufferException**

I have no idea, the received strings are not empty though, they look
like this:
[SPACE]platzhirsch[A Character which cannot be displayed probably]

These character which I cannot read, I also get, when using other
fields like int32. I guess there are just encoded into the string.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To post to this group, send email to prot...@googlegroups.com.
To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.

[platzhirsch]

Right! I think I refactored the whole writing and reading, still I am
stuck. However by this I get this Exception:
com.google.protobuf.UninitializedMessageException: Message missing
required fields: name

In C++:

Name name;
name.set_name("platzhirsch");

boost::asio::streambuf b;
std::ostream os(&b);

ZeroCopyOutputStream *raw_output = new OstreamOutputStream(&os);
CodedOutputStream *coded_output = new CodedOutputStream(raw_output);

coded_output->WriteLittleEndian32(name.ByteSize());
name.SerializeToCodedStream(coded_output);
socket.send(b);

In Java:

NameProtos.Name name =
NameProtos.Name.parseDelimitedFrom(socket.getInputStream());
System.out.println(name.newBuilder().build().toString());


---

I can't find out where the problem might be.

[Jason Hsueh]

On Fri, Apr 15, 2011 at 9:48 AM, platzhirsch <konrad...@googlemail.com> wrote:

Right! I think I refactored the whole writing and reading, still I am
stuck. However by this I get this Exception:
com.google.protobuf.UninitializedMessageException: Message missing
required fields: name

In C++:

Name name;
name.set_name("platzhirsch");

boost::asio::streambuf b;
std::ostream os(&b);

ZeroCopyOutputStream *raw_output = new OstreamOutputStream(&os);
CodedOutputStream *coded_output = new CodedOutputStream(raw_output);

coded_output->WriteLittleEndian32(name.ByteSize());
name.SerializeToCodedStream(coded_output);

You'll need to destroy the CodedOutputStream, which will flush data that's been buffered internally. We should probably make this clearer in the docs.

socket.send(b);

In Java:

NameProtos.Name name =
NameProtos.Name.parseDelimitedFrom(socket.getInputStream());
System.out.println(name.newBuilder().build().toString());


---

I can't find out where the problem might be.

--

[platzhirsch]

I am sorry, could you clarify this? With destroy you mean call the
delete operator? The changes:

boost::asio::streambuf b;
std::ostream os(&b);

ZeroCopyOutputStream *raw_output = new OstreamOutputStream(&os);
CodedOutputStream *coded_output = new CodedOutputStream(raw_output);

coded_output->WriteLittleEndian32(name.ByteSize());
name.SerializeToCodedStream(coded_output);

delete coded_output;
delete raw_output;

socket.send(b);

---

This changes nothing, sadly.

[Jason Hsueh]

yup, that's what I meant. Can you check that you are writing data to the stream? Also, I didn't notice that you are writing the message length as a header. Did you read that value from the socket prior to calling parse? The Java-side parseFrom call should only be reading the data read by SerializeToCodedStream(); you need to consume the WriteLittleEndian32 data separately.

--

[platzhirsch]

Adding the delete operator was good, now actually data it send, as the
socket.send function now returns a value > 0.

I did not read the message length prior to calling parse, but I
changed it:

CodedInputStream cis =
CodedInputStream.newInstance(socket.getInputStream());
cis.readRawLittleEndian32();
NameProtos.Name name = NameProtos.Name.parseFrom(cis);

Sadly, I keep getting:

com.google.protobuf.UninitializedMessageException: Message missing
required fields: name

On Apr 15, 7:03 pm, Jason Hsueh <jas...@google.com> wrote:
> yup, that's what I meant. Can you check that you are writing data to the
> stream? Also, I didn't notice that you are writing the message length as a
> header. Did you read that value from the socket prior to calling parse? The
> Java-side parseFrom call should only be reading the data read by
> SerializeToCodedStream(); you need to consume the WriteLittleEndian32 data
> separately.
>
> On Fri, Apr 15, 2011 at 10:01 AM, platzhirsch

> <konrad.rei...@googlemail.com>wrote:

[platzhirsch]

Oh, maybe this line was flawed after all:

System.out.println(name.newBuilder().build().toString());

Which cannot work, can it? ;-)

I changed it to name.getName()

and there it is, the data I've sent, thank you!

[Jason Hsueh]

Oh, of course, glad it worked.

Also I realized I was mistaken about CodedOutputStream: it doesn't do any internal buffering; it's OstreamOutputStream that does (some other ZeroCopyOutputStream implementations do as well). In any event, keep destroying the streams before you send.

--