[Java][Python][TCP] Reading messages that where written with writeDelimited and viceversa

[Galileo Sanchez]

Hello! I've been searching in this group a little bit, and found this https://groups.google.com/forum/?fromgroups#!searchin/protobuf/python$20delimited/protobuf/_3gAmvU4_80/rEeMA3A9H1UJ

Now, I'm  very very very new to python, "never heard of it" kind of new,  so I got a little confused on how to apply that patch

should I directly patch my compiled request.proto ( that would be request_pb2.py), or should I apply it to some other file that is on the source

of the python implementation of the protocol buffer api.

If it is useful, basically what I'm trying to do is, run a TCP java server, and write clients in python, c++,and as I already have a good

proficiency in java, I wrote the server first, and I used parseDelimitedTo(), and writeDelimitedTo, because it will be reading multiple request from different

clients and that seemed the better way to go. Actually it launches a new thread per client, but anyway, a client could send multiple requests

Just in case that patch isn't intended to the purpose I want, I tried a different approach and tried to write the delimiter, 

str(request.ByteSize())+request.SerializeToString()

but now I see that I need to add the size in a raw bit format

So the questions are:

Is the patch for what i thought it was? 

then              to wich file should I apply it?

else 

if (Should I write the size as a raw bit string?)

then                        How do I do that?

else

How do I write the message in a format that my Java server can understand?

Basically I just don't want to change my server implementation.

Thanks very much people, in the mean time I will be looking for other solutions...

[Evan Jones]

On Mar 25, 2012, at 18:09 , Galileo Sanchez wrote:
> else
> if (Should I write the size as a raw bit string?)
> then How do I do that?

You need to use something like the following. Not 100% sure it works but it should be close? Hope this helps,

Evan

# Output a message to be read with Java's parseDelimitedFrom
import google.protobuf.internal.encoder as encoder
out = message.SerializeToString()
out = encoder._VarintBytes(len(out)) + out

# Read a message from Java's writeDelimitedTo:
import google.protobuf.internal.decoder as decoder

# Read length
(size, position) = decoder._DecodeVarint(buffer, 0)
# Read the message
message_object.ParseFromString(buffer[position:position+size])

--
http://evanjones.ca/

[Galileo Sanchez]

Thanks man... It worked great! I guess I should read the documentation a little more xP.

[Evan Jones]

On Mar 26, 2012, at 21:49 , Galileo Sanchez wrote:
> Thanks man... It worked great! I guess I should read the documentation a little more xP.

Sadly these functions aren't actually documented. The Python API doesn't expose these routines for some reason I don't understand / remember. Glad it worked!

Evan

--
http://evanjones.ca/

[Mike White]

aloha everyone,

i stumbled across this post as i'm hitting this error as well. since i'm a newbie to python i had a question on how to implement this. when doing these steps,

# Read a message from Java's writeDelimitedTo:
import google.protobuf.internal.decoder as decoder

# Read length
(size, position) = decoder._DecodeVarint(buffer, 0)
# Read the message
message_object.ParseFromString(buffer[position:position+size])

i assume i need to set 'buffer' somehow, right? here is what i'm trying and not working...

----------------------------------------------------------

import sys
import ELINT_pb2
import google.protobuf.internal.decoder as decoder

if len(sys.argv) != 2:
    print "Usage: ", sys.argv[0], "<protobuf-file>"
    sys.exit(-1)

elintWrapper = ELINT_pb2.ELINTWrapper()

# Read length
(size, position) = decoder._DecodeVarint(buffer, 0)
# Read the message

#message_object.ParseFromString(buffer[position:position+size])

f = open(sys.argv[1], "rb")
#elintWrapper.ParseFromString(f.read())

elintWrapper.ParseFromString(buffer[position:position+size])
f.close()

----------------------------------------------------------

running it produces this error:

Traceback (most recent call last):
  File "./measurement-reader.py", line 16, in <module>

    (size, position) = decoder._DecodeVarint(buffer, 0)

  File "build/bdist.linux-x86_64/egg/google/protobuf/internal/decoder.py", line 116, in DecodeVarint
TypeError: 'type' object is unsubscriptable

any help would be greatly appreciated!! mahalo everyone!!

[Ilia Mirkin]

On Tue, Sep 29, 2015 at 9:31 AM, Mike White <mik...@gmail.com> wrote:
> aloha everyone,
>
> i stumbled across this post as i'm hitting this error as well. since i'm a
> newbie to python i had a question on how to implement this. when doing these
> steps,
>
> # Read a message from Java's writeDelimitedTo:
> import google.protobuf.internal.decoder as decoder

buffer = open(sys.argv[1], "rb").read()

>
> # Read length
> (size, position) = decoder._DecodeVarint(buffer, 0)
> # Read the message
> message_object.ParseFromString(buffer[position:position+size])

And then this should work, I think.