Type checking messages in Python

[mike....@gmail.com]

When working with gRPC in Python, either implementing a service or client code, it would be great if the types of the messages being returned or passed in were checked. That is, if my service is defined to return a type Foo, but I construct and return a type Bar object, the code should throw an error.

This github issue implies that this is the intended behavior: https://github.com/grpc/grpc/issues/3410

But in fact, I appear to be able to return any Message object without throwing any errors.

Is this a bug? Are there differing opinions on how this should work?

-- 

Mike Bruce

[Nathaniel Manista]

On Tue, Oct 18, 2016 at 11:07 AM, <mike....@gmail.com> wrote:

When working with gRPC in Python,

With what version of gRPC Python are you working, and on what platform? Also with what version of protocol buffers are you working?

either implementing a service or client code, it would be great if the types of the messages being returned or passed in were checked. That is, if my service is defined to return a type Foo, but I construct and return a type Bar object, the code should throw an error.

This github issue implies that this is the intended behavior: https://github.com/grpc/grpc/issues/3410

But in fact, I appear to be able to return any Message object without throwing any errors.

Any message object, or only mutually serializable message objects?

Is this a bug?

No, but if there's an opportunity here to be more helpful to applications that contain defects, we're open to it.

Are there differing opinions on how this should work?

There's almost certainly someone out there who believes that since message serialization is in the RPC path that it shouldn't incur the penalty of explicit type-checking, but that's not my opinion at this time. What I am reluctant to do is include explicit type-checking in generated code, which is where I would think that it would have to go if the message class' serialization method itself doesn't do the type-checking.

-Nathaniel

[Mike Bruce]

On Tue, Oct 18, 2016 at 3:51 PM Nathaniel Manista <nath...@google.com> wrote:

On Tue, Oct 18, 2016 at 11:07 AM, <mike....@gmail.com> wrote:

When working with gRPC in Python,

With what version of gRPC Python are you working, and on what platform? Also with what version of protocol buffers are you working?

gRPC 1.0.0, on Linux and Mac. Protobuf 3.1.0 / proto3.

 

either implementing a service or client code, it would be great if the types of the messages being returned or passed in were checked. That is, if my service is defined to return a type Foo, but I construct and return a type Bar object, the code should throw an error.

This github issue implies that this is the intended behavior: https://github.com/grpc/grpc/issues/3410

But in fact, I appear to be able to return any Message object without throwing any errors.

Any message object, or only mutually serializable message objects?

I've poked around with a lot of different combinations, and haven't gotten anything to fail, as long as it's a Message of some kind. The field types don't have to line up. I'm testing this doing doing the <message class>.SerializeToString(<message object>) that you reference in that github issue.

 

Is this a bug?

No, but if there's an opportunity here to be more helpful to applications that contain defects, we're open to it.

Silently accepting bad data doesn't feel great. One of the things about gRPC that was appealing to us was in having defined types to pass around.

Are there different approaches to achieving a schema-like effect, without writing a bunch of custom code? 

 

Are there differing opinions on how this should work?

There's almost certainly someone out there who believes that since message serialization is in the RPC path that it shouldn't incur the penalty of explicit type-checking, but that's not my opinion at this time. What I am reluctant to do is include explicit type-checking in generated code, which is where I would think that it would have to go if the message class' serialization method itself doesn't do the type-checking.

What's the source of your reluctance? I wouldn't mind putting some work into this if there was a path forward.

 

[Nathaniel Manista]

On Tue, Oct 18, 2016 at 2:06 PM, Mike Bruce <br...@jhereg.net> wrote:

On Tue, Oct 18, 2016 at 3:51 PM Nathaniel Manista <nath...@google.com> wrote:

On Tue, Oct 18, 2016 at 11:07 AM, <mike....@gmail.com> wrote:

When working with gRPC in Python,

With what version of gRPC Python are you working, and on what platform? Also with what version of protocol buffers are you working?

gRPC 1.0.0, on Linux and Mac. Protobuf 3.1.0 / proto3.

Well that checks out.

either implementing a service or client code, it would be great if the types of the messages being returned or passed in were checked. That is, if my service is defined to return a type Foo, but I construct and return a type Bar object, the code should throw an error.

This github issue implies that this is the intended behavior: https://github.com/grpc/grpc/issues/3410

But in fact, I appear to be able to return any Message object without throwing any errors.

Any message object, or only mutually serializable message objects?

I've poked around with a lot of different combinations, and haven't gotten anything to fail, as long as it's a Message of some kind. The field types don't have to line up. I'm testing this doing doing the <message class>.SerializeToString(<message object>) that you reference in that github issue.

 

Is this a bug?

No, but if there's an opportunity here to be more helpful to applications that contain defects, we're open to it.

Silently accepting bad data doesn't feel great. One of the things about gRPC that was appealing to us was in having defined types to pass around.

Agreed.

Are there different approaches to achieving a schema-like effect, without writing a bunch of custom code? 

Are there differing opinions on how this should work?

There's almost certainly someone out there who believes that since message serialization is in the RPC path that it shouldn't incur the penalty of explicit type-checking, but that's not my opinion at this time. What I am reluctant to do is include explicit type-checking in generated code, which is where I would think that it would have to go if the message class' serialization method itself doesn't do the type-checking.

What's the source of your reluctance?

An engineering mindset that says that generated code should be as simple as possible, plus this sort of feeling like a feature that belongs on the protobuf side rather than the gRPC side.

I wouldn't mind putting some work into this if there was a path forward.

Let's start with a bug report to the protobuf folks and see where that takes us?

-N