Python gRPC Endpoints

788 views
Skip to first unread message

mitch....@potential.co.uk

unread,
Sep 21, 2017, 12:27:29 PM9/21/17
to Google Cloud Endpoints
Has anybody else had trouble connecting to a gRPC service that is hosted behind endpoints, with a python client? Service authentication happens over SSL/TLS and the service requires the client to pass an api-key when calling it.

The python gRPC documentation is scant, and mostly demonstrates how to do communication over an insecure channel - great for testing against a locally hosted service, not great for a production system!

My python code creates a secure channel using ssl channel credentials. It passes the api-key in as a metadata array on each RPC call. The response I am getting from the service is: StatusCode.Unavailable, connect failed. Full standard out from client:

 Traceback (most recent call last):
  File "simple_client.py", line 51, in <module>
    run()
  File "simple_client.py", line 44, in run
    response = stub.Attribute(request=request, metadata=metadata)
  File "/usr/local/lib/python2.7/site-packages/grpc/_channel.py", line 492, in __call__
    return _end_unary_response_blocking(state, call, False, deadline)
  File "/usr/local/lib/python2.7/site-packages/grpc/_channel.py", line 440, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>


For reference, there is nothing wrong with my service; I have been able to create a Go client that can successfully communicate with it.

Attached are my python and go clients (with the api-key obfuscated). Any help would be greatly appreciated.

Cheers,
Mitch
go_client.py
python_client.py

Lizan Zhou

unread,
Sep 21, 2017, 2:44:33 PM9/21/17
to mitch....@potential.co.uk, Google Cloud Endpoints
The error logs says "Connect Failed" so I guess maybe it is because certificate verification failed.

Did you try use creds with non-default root CA? like:
grpc.ssl_channel_credentials(open('roots.pem').read())

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/21500aa1-52f8-4a2d-ae6a-117884bac0b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

mitch....@potential.co.uk

unread,
Sep 22, 2017, 4:38:09 AM9/22/17
to Google Cloud Endpoints
Thanks for your reply Lizan.

No I haven't tried that. Where can I get another roots.pem file from?

Lizan Zhou

unread,
Sep 22, 2017, 11:59:26 AM9/22/17
to mitch....@potential.co.uk, Google Cloud Endpoints
That would be your certificate chain you deployed to Endpoints.

Or you can try what Golang trust by default:

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

mitch....@potential.co.uk

unread,
Sep 22, 2017, 12:50:07 PM9/22/17
to Google Cloud Endpoints
Thanks... I didn't deploy the service to endpoints myself, so didn't have the certificate chain. However, I spoke to the person who did, and they have given me the cert files. Unfortunately, still no success.

Could the problem stem from the an error in how python's gRPC client communicates wtih endpoints?
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/21500aa1-52f8-4a2d-ae6a-117884bac0b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Lizan Zhou

unread,
Sep 22, 2017, 4:04:54 PM9/22/17
to mitch....@potential.co.uk, Google Cloud Endpoints
On Fri, Sep 22, 2017 at 9:50 AM, <mitch....@potential.co.uk> wrote:
Thanks... I didn't deploy the service to endpoints myself, so didn't have the certificate chain. However, I spoke to the person who did, and they have given me the cert files. Unfortunately, still no success.

Could the problem stem from the an error in how python's gRPC client communicates wtih endpoints?
Can you try to set GRPC_VERBOSITY env vars and see if there are additional logs?

What is the gRPC version are you using in Python?
 
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

Lizan Zhou

unread,
Sep 22, 2017, 4:14:45 PM9/22/17
to mitch....@potential.co.uk, Google Cloud Endpoints
It might be this gRPC issue: https://github.com/grpc/grpc/issues/9987

mitch....@potential.co.uk

unread,
Sep 25, 2017, 5:30:53 AM9/25/17
to Google Cloud Endpoints
Hi,

I am using gRPC version: 1.6.0.

Thanks for your suggestions. I have added GRPC_VERBOSITY and GRPC_TRACE environment variables.This line from the output looks interesting: 

D0925 10:00:13.613308000 123145561743360 security_handshaker.c:111]    Security handshake failed: {"created":"@1506330013.613296000","description":"Cannot check peer: missing selected ALPN property.","file":"src/core/lib/security/transport/security_connector.c","file_line":600}

Apparently, some of my colleagues have encountered a similar issue when writing Java clients to connect to gRPC services via endpoints. It seems the problem is that Google's load balancer in GCP does not support ALPN. How do google get around this problem internally? 

Cheers,
Mitch
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

Lizan Zhou

unread,
Sep 26, 2017, 5:02:32 PM9/26/17
to mitch....@potential.co.uk, Google Cloud Endpoints
Which platform are you on and how is your load balancer configured? If it's load balancer issue, I need more information to debug it.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages