Can't read in Java message sent from C++

[Maxim Leonovich]

Few days I'm trying to send a message from C++ to Java over the
network but unsuccessfully.
I needed to send a several messages over one connection. I'm doing
this:

//C++ side

....
void MainWindow::client_connected() {
GPSChatMessage msg = GPSChatMessage();
msg.set_type(proto::GPSChatMessage::AUTHORIZATION_REQUEST);
msg.mutable_auth_req_msg()->set_login("bsod");
msg.mutable_auth_req_msg()->set_password("*****");
client->send(msg);
}
....
....
void Client::send(const GPSChatMessage & msg) {
QDataStream to(socket);
char * buffer = new char[msg.ByteSize() + 4];
ZeroCopyOutputStream * os = new
ArrayOutputStream(buffer,msg.ByteSize() + 4,sizeof(char));
CodedOutputStream * cos = new CodedOutputStream(os);
cos->WriteLittleEndian32(msg.ByteSize()); //Tryed
"WriteVariant32", didn't help
msg.SerializeToCodedStream(cos);
delete cos;
delete os;
to.writeBytes(buffer,msg.ByteSize() + 4);
delete buffer;
}

//Java side:
//(I have a thread, that represents user connection. "from" - is an
InputStream from socket.

public void run() {
while (!interrupted()) {
try {
GPSChatMessage msg = GPSChatMessage.parseDelimitedFrom(from);
handler.handle(msg);
} catch (IOException ex) {
Logger.getLogger(ClientThread.class.getName()).log(Level.SEVERE,
null, ex);
}


When I sending a message from C++, I'm getting 4 exceptions in Java:

10.07.2010 14:30:42 net.ClientThread run
SEVERE: null
com.google.protobuf.InvalidProtocolBufferException: Message missing
required fields: type
at
com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(Unknown
Source)
at proto.GPSChatProtocol$GPSChatMessage
$Builder.buildParsed(GPSChatProtocol.java:299)
at proto.GPSChatProtocol$GPSChatMessage$Builder.access
$200(GPSChatProtocol.java:247)
at proto.GPSChatProtocol
$GPSChatMessage.parseDelimitedFrom(GPSChatProtocol.java:218)
at net.ClientThread.run(ClientThread.java:54)
10.07.2010 14:30:43 net.ClientThread run
SEVERE: null
com.google.protobuf.InvalidProtocolBufferException: Message missing
required fields: type
at
com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(Unknown
Source)
at proto.GPSChatProtocol$GPSChatMessage
$Builder.buildParsed(GPSChatProtocol.java:299)
at proto.GPSChatProtocol$GPSChatMessage$Builder.access
$200(GPSChatProtocol.java:247)
at proto.GPSChatProtocol
$GPSChatMessage.parseDelimitedFrom(GPSChatProtocol.java:218)
at net.ClientThread.run(ClientThread.java:54)
10.07.2010 14:30:43 net.ClientThread run
SEVERE: null
com.google.protobuf.InvalidProtocolBufferException: Message missing
required fields: type
at
com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(Unknown
Source)
at proto.GPSChatProtocol$GPSChatMessage
$Builder.buildParsed(GPSChatProtocol.java:299)
at proto.GPSChatProtocol$GPSChatMessage$Builder.access
$200(GPSChatProtocol.java:247)
at proto.GPSChatProtocol
$GPSChatMessage.parseDelimitedFrom(GPSChatProtocol.java:218)
at net.ClientThread.run(ClientThread.java:54)
10.07.2010 14:30:43 net.ClientThread run
SEVERE: null
com.google.protobuf.InvalidProtocolBufferException: Protocol message
end-group tag did not match expected tag.
at
com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(Unknown
Source)
at
com.google.protobuf.CodedInputStream.checkLastTagWas(Unknown Source)
at com.google.protobuf.AbstractMessageLite
$Builder.mergeFrom(Unknown Source)
at com.google.protobuf.AbstractMessage
$Builder.mergeFrom(Unknown Source)
at com.google.protobuf.AbstractMessage
$Builder.mergeFrom(Unknown Source)
at com.google.protobuf.AbstractMessageLite
$Builder.mergeDelimitedFrom(Unknown Source)
at com.google.protobuf.AbstractMessage
$Builder.mergeDelimitedFrom(Unknown Source)
at proto.GPSChatProtocol
$GPSChatMessage.parseDelimitedFrom(GPSChatProtocol.java:218)
at net.ClientThread.run(ClientThread.java:54)

I'm googling a lot, but I really don't know what to do. What I'm doing
wrong?

[jamesmi...@googlemail.com]

I encountered a similar problem with reading the bytes in C++ from
data written in Java.
Solved it by swapping the bytes :
uint32_t magic_number = ntohl(magic_number_);

http://github.com/h4ck3rm1k3/OSM-Osmosis/commit/714be97af12698f83152d2c1d0e407337e82803b

mike

> --
> You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
> To post to this group, send email to prot...@googlegroups.com.
> To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
>
>

--
James Michael DuPont
Member of Free Libre Open Source Software Kosova and Albania
flossk.org flossal.org

[Evan Jones]

On Jul 10, 2010, at 7:47 , Maxim Leonovich wrote:
> ArrayOutputStream(buffer,msg.ByteSize() + 4,sizeof(char));

The documentation states:

block_size is mainly useful for testing; in production you would
probably never want to set it.

So you should get rid of the "sizeof(char)" part.

> cos->WriteLittleEndian32(msg.ByteSize()); //Tryed
> "WriteVariant32", didn't help
> msg.SerializeToCodedStream(cos);

If you want to use Java's .parseDelimitedFrom, you *must* use
WriteVarint32, because that is the format it expects the length
prefix. In this case, you'll need to call ArrayOutputStream::
ByteCount() to figure out how many bytes were actually serialized.

You also probably should create the ArrayOutputStream and
CodedOutputStream on the stack, rather than using new. This will be
slightly faster.

That said, the only issue here that affects correctness is the
WriteVarint32 part. The rest shouldn't matter unless I missed
something. You should change your code to do that, then if you are
still having problems you should try dumping the contents of the
buffer on both the C++ and the Java side. Maybe the input/output is
getting messed up somewhere?

Good luck,

Evan

--
Evan Jones
http://evanjones.ca/

[Maxim Leonovich]

I watched on both buffers: they are similar, but in C++ I can't find a
leading Variant32 with size, when in Java it exists. The rests of
buffers are identical.
http://pic4.ru/8337
http://pic4.ru/8338
http://pic4.ru/8339

[jamesmi...@googlemail.com]

the leading variant in c++ is most likely in the wrong byte order if
you are running a x86 machine. as i said, you need to flip them.
mike

[Maxim Leonovich]

I tried to convert msg.ByteSize() to LittleEndian and to BigEndian,
but that didn't helps.
I thought, that {0 0 0 18 ... } is the size, but no. Size is 17.
Look at screens: in C++ I have {17 8 1 26 12 ...}. In Java : {0 0 0 18
17 8 1 26 12...}
But what is {0 0 0 18} ? I have no more IO operations exept in
Client::send().

On 10 июл, 22:00, "jamesmikedup...@googlemail.com"

> > For more options, visit this group athttp://groups.google.com/group/protobuf?hl=en.

[jamesmi...@googlemail.com]

Ok, well then just read out 4 bytes first in java and they get your 17 8 1 ....

0 0 0 18 might be the lenght that you are not reading first.
mike

> For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.

[Kenton Varda]

On Sat, Jul 10, 2010 at 11:53 AM, Maxim Leonovich <lm....@gmail.com> wrote:

I watched on both buffers: they are similar, but in C++ I can't find a
leading Variant32 with size, when in Java it exists. The rests of
buffers are identical.
http://pic4.ru/8337
http://pic4.ru/8338
http://pic4.ru/8339

0 0 0 18 is NOT a varint.  I think that something else is adding this to the front of your buffer on the C++ end -- these bytes are NOT coming from protocol buffers, as your screenshots plainly demonstrate.

The 17 appears to be the varint size (it is only one byte in this case).

 

[Kenton Varda]

On Sat, Jul 10, 2010 at 12:00 PM, jamesmi...@googlemail.com <jamesmi...@googlemail.com> wrote:

the leading variant in c++ is most likely in the wrong byte order if
you are running a x86 machine. as i said, you need to flip them.
mike

Mike, varint (I assume that's what you meant by "variant") is a completely different, variable-width integer encoding.  It is not affected by the machine's byte order.

Furthermore, Protocol Buffers handles all endianness issues internally, so if you are using Coded{Input,Output}Stream exclusively, you can't have byte ordering issues.

[Maxim Leonovich]

Thanks for all. I figured out. The problem with "{0 0 0 18}" really
was in Qt library, not in protobuffs.

On 11 июл, 07:53, Kenton Varda <ken...@google.com> wrote:
> On Sat, Jul 10, 2010 at 12:00 PM, jamesmikedup...@googlemail.com <

>
> jamesmikedup...@googlemail.com> wrote:
> > the leading variant in c++ is most likely in the wrong byte order if
> > you are running a x86 machine. as i said, you need to flip them.
> > mike
>
> Mike, varint (I assume that's what you meant by "variant") is a completely
> different, variable-width integer encoding.  It is not affected by the
> machine's byte order.
>
> Furthermore, Protocol Buffers handles all endianness issues internally, so
> if you are using Coded{Input,Output}Stream exclusively, you can't have byte
> ordering issues.
>
>
>
>
>

> > On Sat, Jul 10, 2010 at 8:53 PM, Maxim Leonovich <lm.b...@gmail.com>

> > protobuf+u...@googlegroups.com<protobuf%2Bunsubscribe@googlegroups.c om>
> > .

> > > For more options, visit this group at
> >http://groups.google.com/group/protobuf?hl=en.
>
> > --
> > James Michael DuPont
> > Member of Free Libre Open Source Software Kosova and Albania
> > flossk.org flossal.org
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Protocol Buffers" group.
> > To post to this group, send email to prot...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > protobuf+u...@googlegroups.com<protobuf%2Bunsubscribe@googlegroups.c om>
> > .