[GSoC] Assistance with gRPC Python Ideas

[Sword Chen]

Hi all,

I am Sword Chen, a postgraduate student at Mid Sweden University. I have used the gPRC for some time. So I would love to contribute to the project as part of GSoC. I have seen the given ideas, and I am quite interested in the "Support static type-checking of both gRPC Python" , but I can't find the relative open issue for this idea.  so I did some prework and I put them here.

I have tried the Type Checking Tool Mypy, when I tested some file using the grpc module and I got this error " Cannot find module named 'grpc' " Then I went through with the Mypy document, Mypy can't do static type-checking of both gRPC Python itself and of the code that uses gRPC because Mypy can't find a stub for a grpc library module. and here is the information about Creating Stubs For Python Modules https://github.com/python/mypy/wiki/Creating-Stubs-For-Python-Modules. So what I need to do is to create the stubs for the grpc to help Mypy to detect the code. Am I working in the right direction now? It would be really helpful if someone can give me some guidance, so I can have some confidence to go on.

Thanks, 

Sword

[Nathaniel Manista]

On Sat, Mar 10, 2018 at 11:07 AM, Sword Chen <titan...@gmail.com> wrote:

I am Sword Chen, a postgraduate student at Mid Sweden University.

Pleased to meet you!

I have used the gPRC for some time.

Cool! How long? With what programming languages?

So I would love to contribute to the project as part of GSoC. I have seen the given ideas, and I am quite interested in the "Support static type-checking of both gRPC Python" , but I can't find the relative open issue for this idea.  so I did some prework and I put them here.

I have tried the Type Checking Tool Mypy, when I tested some file using the grpc module and I got this error " Cannot find module named 'grpc' " Then I went through with the Mypy document, Mypy can't do static type-checking of both gRPC Python itself and of the code that uses gRPC because Mypy can't find a stub for a grpc library module. and here is the information about Creating Stubs For Python Modules https://github.com/python/mypy/wiki/Creating-Stubs-For-Python-Modules. So what I need to do is to create the stubs for the grpc to help Mypy to detect the code. Am I working in the right direction now? It would be really helpful if someone can give me some guidance, so I can have some confidence to go on.

This sounds like it's going in the right direction - something that I think we sort of expect is that there will be a certain incrementality to static typing of gRPC Python, and having to add stubs describing the types of lower-level systems so that the upper-level systems can have type annotations added and checked is not a surprise. mypy and pytype are both young enough, and gRPC Python is weird enough, that it is certainly not our expectation that we can just throw either tool at our codebase and it will figure types out without any further assistance. :-)

Thanks for reaching out!

-Nathaniel

[Sword Chen]

Thanks for your reply.

Cool! How long? With what programming languages?

 I have used gRPC for several weeks, and I have been building the stream algorithm test environment these weeks with gRPC in Python.  

This sounds like it's going in the right direction - something that I think we sort of expect is that there will be a certain incrementality to static typing of gRPC Python, and having to add stubs describing the types of lower-level systems so that the upper-level systems can have type annotations added and checked is not a surprise. mypy and pytype are both young enough, and gRPC Python is weird enough, that it is certainly not our expectation that we can just throw either tool at our codebase and it will figure types out without any further assistance. :-)

 You mean the gradual typing, which allows one to annotate only part of a program, thus leverage desirable aspects of both dynamic and static typing. PEP483.  If we don't use the other tools, I have tried some basic type check by using annotations and decorator PEP 3107, here is the decorator function which can do the type check:

from inspect import signature
from functools import wraps


def typeassert(*ty_args, **ty_kwargs):
    def decorate(func):


        # Map function argument names to supplied types
        sig = signature(func)
        bound_types = sig.bind_partial(*ty_args, **ty_kwargs).arguments


        @wraps(func)
        def wrapper(*args, **kwargs):
            bound_values = sig.bind(*args, **kwargs)
            # Enforce type assertions across supplied arguments
            for name, value in bound_values.arguments.items():
                if name in bound_types:
                    if not isinstance(value, bound_types[name]):
                        raise TypeError(
                            'Argument {} must be {}'.format(name, bound_types[name])
                            )
            return func(*args, **kwargs)
        return wrapper
    return decorate

Here is the example:

@typeassert(int,int z=int)
def spam(x, y, z=42):
    print(x, y, z)

So as you said, if we don't throw other tools into the codebase, I am considering that we could use this method to help do the type check. But I still have some confusions:

1.  the idea is  "Support static type-checking of both gRPC Python itself and of code that uses gRPC Python."  when we use the gPRC python, we can add the decorator to our own function, but for gPRC Python itself, do we add type-checker to every defined function? this will 

2.  Does this need to support PEP 484 or just detect the errors using the gRPC Python? As I know, mypy verifies standard PEP 484 type hints. 

3. About the grpc structure, actually, I am not quite Clear, is there some document to help me understand it? 

These are my thoughts. Hope to get further guidance.

- Sword

在 2018年3月13日星期二 UTC+8上午3:59:15，Nathaniel Manista写道：

[Nathaniel Manista]

On Wed, Mar 14, 2018 at 8:06 AM, Sword Chen <titan...@gmail.com> wrote:

Thanks for your reply.

Cool! How long? With what programming languages?

 I have used gRPC for several weeks, and I have been building the stream algorithm test environment these weeks with gRPC in Python.  

This sounds like it's going in the right direction - something that I think we sort of expect is that there will be a certain incrementality to static typing of gRPC Python, and having to add stubs describing the types of lower-level systems so that the upper-level systems can have type annotations added and checked is not a surprise. mypy and pytype are both young enough, and gRPC Python is weird enough, that it is certainly not our expectation that we can just throw either tool at our codebase and it will figure types out without any further assistance. :-)

 You mean the gradual typing, which allows one to annotate only part of a program, thus leverage desirable aspects of both dynamic and static typing.

Well... I don't think there are desirable aspects of dynamic typing, but not everyone agrees with me. :-P

Huh. I actually don't know what the relationship is between 3107 and 483/484, but... we definitely don't want to add dynamic type checks. gRPC needs to be fast; correct code shouldn't have to pay a performance penalty at run-time.

So as you said, if we don't throw other tools into the codebase, I am considering that we could use this method to help do the type check. But I still have some confusions:

1.  the idea is  "Support static type-checking of both gRPC Python itself and of code that uses gRPC Python."  when we use the gPRC python, we can add the decorator to our own function, but for gPRC Python itself, do we add type-checker to every defined function? this will 

2.  Does this need to support PEP 484 or just detect the errors using the gRPC Python? As I know, mypy verifies standard PEP 484 type hints.

Yes to supporting what are called "type hints" and defined in 484.

3. About the grpc structure, actually, I am not quite Clear, is there some document to help me understand it?

There's not enough! Building, running, and testing the code is probably the best way to get to know it (especially changing parts here and there and seeing what happens). We also have documentation for the gRPC Python API.

These are my thoughts. Hope to get further guidance.

Good to hear from you,

-N