How to read JSON into grpc python message objects? How to output JSON from python?

[Zachary Deretsky]

I have JSON files generated from C++ code by BinaryToJsonString() calls.

How to import them into a client written in Python?

Thanks, Zach.

[Nathaniel Manista]

On Fri, Jul 1, 2016 at 3:50 AM, Zachary Deretsky <zder...@gmail.com> wrote:

I have JSON files generated from C++ code by BinaryToJsonString() calls.

How to import them into a client written in Python?

I don't think I fully understand your question. Might you share some code - either the code you've written so far, or the code you'd like to be able to write to solve your problem, or both?

-Nathaniel

[Zachary Deretsky]

Nathaniel, thank you.

I am attaching the code with README instructions.

In my project the client creates and edits data files via a browser, so these files are json files.

They are received by a C++ server and this part is working.

I also need to use this data in python clients.

I know, I can create workarounds by carrying serialization to string, but grpc docs claim support for json and I'd prefer to avoid complexity of data duplication.

Regards, Zach.

[Nathaniel Manista]

On Sat, Jul 2, 2016 at 11:35 PM, Zachary Deretsky <zder...@gmail.com> wrote:

I am attaching the code with README instructions.

In my project the client creates and edits data files via a browser, so these files are json files.

They are received by a C++ server and this part is working.

I also need to use this data in python clients.

Your services.proto file contains only messages, not services - is this intended? Having no services defined means that when you run services.sh (which itself runs protoc with the gRPC Python plugin) the _pb2.py file that is generated will have a bunch of gRPC-related imports but not actually define any application-exposed functions and classes (this is the case in the _pb2.py file included in your .tgz).

From the look of your generated code I suspect you're using gRPC 0.14.x - if possible, please upgrade to 0.15.x.

I know, I can create workarounds by carrying serialization to string, but grpc docs claim support for json and I'd prefer to avoid complexity of data duplication.

Please link me to the particular docs you're reading? I'm not personally up to speed on the JSON side of gRPC; I apologize that you might have to wait until after the long holiday weekend for fully-informed answers to your questions.

-N

[Zachary Deretsky]

Nathaniel,

I created a smallest testcase to demonstrate the relevant issue.

My real application runs via browser <-> jinx <-> node.js grpc client <-> C++ grpc server in the cloud.

It also uses Google Map API.

My C++  grpc client also works, I need to use Python to create regression harness.

I the last time built grpc from the master branch on June 18 as suggested here:

http://www.grpc.io/docs/#get-the-source-code

I installed python code as suggested by Ken here  (but not July 1 version yet):

https://groups.google.com/forum/#!topic/grpc-io/4MguzwPvAb8

Commitment to JSON is documented here:

https://developers.google.com/protocol-buffers/docs/proto3#services 

I attach my real .proto file to show that I do use services.

Thanks, Zach.

[Nathaniel Manista]

On Sun, Jul 3, 2016 at 10:30 PM, Zachary Deretsky <zder...@gmail.com> wrote:

I created a smallest testcase to demonstrate the relevant issue.

My real application runs via browser <-> jinx <-> node.js grpc client <-> C++ grpc server in the cloud.

It also uses Google Map API.

My C++  grpc client also works, I need to use Python to create regression harness.

I'm still unclear on the specific task that is an obstacle for you but this detail helps - is it that you are trying to write the Python equivalent of the "services_json.cpp" file included in your .tgz? In your .tgz I don't think I'm seeing a Python file that is your application code - could you share that? Even if it's not yet working or includes pseudocode bits like "# Here I would like to be able to call a function that takes my_first_object and my_second_object and returns a services_pb2.MedServicesArray with values set according to those found in my_first_object and my_second_object.", I think it would be illustrative to see more specifically what you're trying to do in Python.

Commitment to JSON is documented here:

https://developers.google.com/protocol-buffers/docs/proto3#services

Thanks for this; I hadn't seen it before.

-N

[Zachary Deretsky]

Nathaniel,

my services.py is included, please see below.

I need to populate the variable ma from the file services.json

Regards, Zach.

#!/usr/bin/env python

from __future__ import print_function

from grpc.beta import implementations

import services_pb2

_TIMEOUT_SECONDS = 10

def run():

  m = services_pb2.MedService(service_idx = 15, id = "Some_UIID", name = "Sample", available = 85)

  print(dir(m))

  print(m.ListFields())

  ma = services_pb2.MedServiceArray()

# Need to fill-in ma from the file services.json 

if __name__ == '__main__':

  run()

[xiao...@google.com]

If you want to convert a JSON file to a protobuf message, you can use the json_format utility provided by protobuf library:

https://github.com/google/protobuf/blob/master/python/google/protobuf/json_format.py#L308

[Zachary Deretsky]

Thank you,

this can read camelCase json data.