Python no-codegen API

227 views
Skip to first unread message

Riccardo Fiorista

unread,
Jun 29, 2021, 3:58:51 AM6/29/21
to grpc.io
Hi there!

Excited to be part of the gRPC community.
I've recently posted a feature request to the 

However, I'm actually facing some issues atm and am wondering if I'm on a completely wrong track here.

We're currently trying to move our python code-base to the experimental no-codegen API using the commodity function:
```
grpc.protos_and_services(protobuf_path)
``` 
There's also a very basic example of the usage (here in the python examples)[https://github.com/grpc/grpc/tree/master/examples/python/no_codegen]. 

However, we're currently facing the issues that protobuf files importing other protobuf files require special treatment which we addressed like this:
```
# load.py
import shutil
import grpc
import os


def get_protos_and_services(service):
    file_paths = _import_proto_file(service)
    pb2, pb2_grpc = grpc.protos_and_services(file_paths[0])
    # need to do it this way as tempfile doesn't work
    for f in file_paths:
        os.remove(f)
    return pb2, pb2_grpc


def _import_proto_file(service):
    """
    Recursively imports all necessary protofiles
    """
    file_paths = [f"{service}.proto"]
    with open(file_paths[0], 'w+b') as f:
        shutil.copy2(_get_absolute_protobuf_file_path(service), f.name)
        for line in f:
            l = line.decode('utf-8')
            if 'import "' in l:
                service_to_import = l.replace(' ', '').lstrip('import"').split('.')[0]
                file_paths.extend(_import_proto_file(service_to_import))
    return file_paths

def _get_absolute_protobuf_file_path(service):
    return os.path.dirname(os.path.abspath(__file__)) + f'/{service}.proto'
```
While this is far from elegant, it works. 

Now, there's another protobuf file that has an import from the official google standard protobuf lib:
```
import "google/protobuf/timestamp.proto";
```
This case is not covered from the code above and we're currently struggling to find a solution of how to import it using the no-codegen API.

Anyone an idea what we could do here?

Cheers,
Riccardo

Amit Saha

unread,
Jun 29, 2021, 11:47:22 PM6/29/21
to Riccardo Fiorista, grpc.io
I couldn't find any resource online talking about no_codegen - (if you
have any handy, that would be great). However, perhaps you could clone
the repository https://github.com/protocolbuffers/protobuf/ and then
read the
directory tree/master/src/google/protobuf looking for the imports?


>
> Cheers,
> Riccardo
>
> --
> You received this message because you are subscribed to the Google Groups "grpc.io" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/e5006763-fee8-4f0c-9fa1-10c75d13d8b8n%40googlegroups.com.

Richard Belleville

unread,
Jul 7, 2021, 1:09:31 PM7/7/21
to grpc.io

Responded in [the corresponding Github issue](https://github.com/grpc/grpc/issues/26571#issuecomment-875763020).
Reply all
Reply to author
Forward
0 new messages