Python gRPC Endpoints

[mitch....@potential.co.uk]

Has anybody else had trouble connecting to a gRPC service that is hosted behind endpoints, with a python client? Service authentication happens over SSL/TLS and the service requires the client to pass an api-key when calling it.

The python gRPC documentation is scant, and mostly demonstrates how to do communication over an insecure channel - great for testing against a locally hosted service, not great for a production system!

My python code creates a secure channel using ssl channel credentials. It passes the api-key in as a metadata array on each RPC call. The response I am getting from the service is: StatusCode.Unavailable, connect failed. Full standard out from client:

 Traceback (most recent call last):
  File "simple_client.py", line 51, in <module>
    run()
  File "simple_client.py", line 44, in run
    response = stub.Attribute(request=request, metadata=metadata)
  File "/usr/local/lib/python2.7/site-packages/grpc/_channel.py", line 492, in __call__
    return _end_unary_response_blocking(state, call, False, deadline)
  File "/usr/local/lib/python2.7/site-packages/grpc/_channel.py", line 440, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Connect Failed)>

For reference, there is nothing wrong with my service; I have been able to create a Go client that can successfully communicate with it.

Attached are my python and go clients (with the api-key obfuscated). Any help would be greatly appreciated.

Cheers,

Mitch

[Lizan Zhou]

The error logs says "Connect Failed" so I guess maybe it is because certificate verification failed.

Did you try use creds with non-default root CA? like:

grpc.ssl_channel_credentials(open('roots.pem').read())

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/21500aa1-52f8-4a2d-ae6a-117884bac0b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[mitch....@potential.co.uk]

Thanks for your reply Lizan.

No I haven't tried that. Where can I get another roots.pem file from?

[Lizan Zhou]

That would be your certificate chain you deployed to Endpoints.

Or you can try what Golang trust by default:

https://golang.org/src/crypto/x509/root_linux.go

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/21500aa1-52f8-4a2d-ae6a-117884bac0b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/b4d9e1ae-bdd0-49a0-8548-cf8991ebc662%40googlegroups.com.

[mitch....@potential.co.uk]

Thanks... I didn't deploy the service to endpoints myself, so didn't have the certificate chain. However, I spoke to the person who did, and they have given me the cert files. Unfortunately, still no success.

Could the problem stem from the an error in how python's gRPC client communicates wtih endpoints?

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/21500aa1-52f8-4a2d-ae6a-117884bac0b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lizan Zhou]

On Fri, Sep 22, 2017 at 9:50 AM, <mitch....@potential.co.uk> wrote:

Thanks... I didn't deploy the service to endpoints myself, so didn't have the certificate chain. However, I spoke to the person who did, and they have given me the cert files. Unfortunately, still no success.

Could the problem stem from the an error in how python's gRPC client communicates wtih endpoints?

Can you try to set GRPC_VERBOSITY env vars and see if there are additional logs?

https://github.com/grpc/grpc/blob/master/doc/environment_variables.md

What is the gRPC version are you using in Python?

 

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/21500aa1-52f8-4a2d-ae6a-117884bac0b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/b4d9e1ae-bdd0-49a0-8548-cf8991ebc662%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/f5ab4ff7-046f-4545-8ec4-7f9cf49ee725%40googlegroups.com.

[Lizan Zhou]

It might be this gRPC issue: https://github.com/grpc/grpc/issues/9987

[mitch....@potential.co.uk]

Hi,

I am using gRPC version: 1.6.0.

Thanks for your suggestions. I have added GRPC_VERBOSITY and GRPC_TRACE environment variables.This line from the output looks interesting: 

D0925 10:00:13.613308000 123145561743360 security_handshaker.c:111]    Security handshake failed: {"created":"@1506330013.613296000","description":"Cannot check peer: missing selected ALPN property.","file":"src/core/lib/security/transport/security_connector.c","file_line":600}

Apparently, some of my colleagues have encountered a similar issue when writing Java clients to connect to gRPC services via endpoints. It seems the problem is that Google's load balancer in GCP does not support ALPN. How do google get around this problem internally? 

Cheers,

Mitch

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/21500aa1-52f8-4a2d-ae6a-117884bac0b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/b4d9e1ae-bdd0-49a0-8548-cf8991ebc662%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

[Lizan Zhou]

Which platform are you on and how is your load balancer configured? If it's load balancer issue, I need more information to debug it.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/21500aa1-52f8-4a2d-ae6a-117884bac0b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/b4d9e1ae-bdd0-49a0-8548-cf8991ebc662%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsubscri...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/f5ab4ff7-046f-4545-8ec4-7f9cf49ee725%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/c3285f2f-4f00-4a60-9842-17a57350487e%40googlegroups.com.