Cloud Endpoints for Compute Engine with ESPv2 || SSL Issue

[Ankit Dwivedi]

Hi Friends,

I have done the setup according to the giving instructions in the document and its running fine without SSL, but when I am doing SSL configure its not working. I have followed below given steps, can you please help me to understand, what wrong I am doing.

1. Uncomment the line no 21( # -https) from openapi.yaml

2. create cert and key using below-

    openssl req -x509 -nodes -days 365 -newkey rsa:2048 \ -keyout ./server.key -out ./server.crt

3. place in directory - 

   sudo mkdir -p /etc/esp/ssl

  sudo cp server.* /etc/esp/ssl/

4. deploy with docker 

 sudo docker run --name=esp \

 --detach \

 --publish=443:9000 \

 --net=esp_net \

 --volume=/etc/esp/ssl:/etc/esp/ssl \

  gcr.io/endpoints-release/endpoints-runtime:2 \

 --service=echo-api.endpoints.*********.cloud.goog \

 --rollout_strategy=managed \

 --backend=echo:8080 \

 --ssl_server_cert_path=/etc/esp/ssl \

 --listener_port=9000

5. Curl request - 

curl -k -d '{"message":"hello world"}' -H "content-type:application/json" https://echo-api.endpoints.*********.cloud.goog:443/echo?key=************

 

all the above operations executing successfully but when hitting the api not getting the response.

kindly help me to understand, what wrong I am doing.

[Wayne Zhang]

What `CN` or `subjectAltName` did you set in the certificate?   It has to match with your server domain name. 

Otherwise, you may have to debug it.   You can use "curl -v" to see what ssl error

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/be73b5cf-86a6-4758-a8b0-30b77cb00074n%40googlegroups.com.

[Ankit Dwivedi]

Yes I am using the same, " echo-api.endpoints.*********.cloud.goog   " this is my fqdns name and I am using same when generating the certificate.

Please check below- 

deyes@deyesinstance:~$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 \

> -keyout ./server.key -out ./server.crt

Generating a RSA private key

...............................................................................................+++++

..................................+++++

writing new private key to './server.key'

-----

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [AU]:IN

State or Province Name (full name) [Some-State]:HR

Locality Name (eg, city) []:GUR

Organization Name (eg, company) [Internet Widgits Pty Ltd]:ELE

Organizational Unit Name (eg, section) []:ELEV

Common Name (e.g. server FQDN or YOUR name) []:echo-api.endpoints.deyes-295406.cloud.goog

Email Address []:ankit...@gmail.com

[Wayne Zhang]

Does your GCE VM have an external IP?    Did you associate DNS echo-api.endpoints.deyes-295406.cloud.goog with that IP address?

Could you try to test it in the GCE VM with localhost first?   Try to run "curl" inside the VM.  curl https://localhost/echo.   For localhost testing, you may have to re-generate the certificate with "localhost" domain name

Well,   "curl -v"  should show errors,  what error did you get?

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/c206a736-4421-4012-aa6f-44fec9fa2ac7n%40googlegroups.com.

[Ankit Dwivedi]

yes I associated the dns with the external ip address, please check below the dns setting.

to check with localhost I created new certificates with 'localhost' as dns name and fired the curl requests from instance ssh terminal and response is an error as given below.

request - 

 curl https://localhost/echo

error res- curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to localhost:443

Request-2- curl -k -d '{"message":"hello world"}' -H "content-type:application/json" https://localhost/echo?key=AIzaSyB1MV_bEbTsthB19clRLfjJHsSY8tIPMTI

error res- error res- curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to localhost:443

request-3-  curl -vv -d '{"message":"foo"}' -H 'Content-Type: application/json' "https://localhost:443/echo?key=AIzaSyB1MV_bEbTsthB19clRLfjJHsSY8tIPMTI"

res-   

* Expire in 0 ms for 6 (transfer 0x564d5f4f4f50)

* Expire in 1 ms for 1 (transfer 0x564d5f4f4f50)

* Expire in 0 ms for 1 (transfer 0x564d5f4f4f50)

* Expire in 1 ms for 1 (transfer 0x564d5f4f4f50)

* Expire in 0 ms for 1 (transfer 0x564d5f4f4f50)

* Expire in 0 ms for 1 (transfer 0x564d5f4f4f50)

* Expire in 0 ms for 1 (transfer 0x564d5f4f4f50)

*   Trying ::1...

* TCP_NODELAY set

* Expire in 149999 ms for 3 (transfer 0x564d5f4f4f50)

* Expire in 200 ms for 4 (transfer 0x564d5f4f4f50)

* Connected to localhost (::1) port 443 (#0)

* ALPN, offering h2

* ALPN, offering http/1.1

* successfully set certificate verify locations:

*   CAfile: none

  CApath: /etc/ssl/certs

* TLSv1.3 (OUT), TLS handshake, Client hello (1):

* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to localhost:443

* Closing connection 0

curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to localhost:443

[Ankit Dwivedi]

and this is the response of curl -v and curl -V.

deyes@deyesinstance:~$ curl -v

curl: no URL specified!

curl: try 'curl --help' or 'curl --manual' for more information

deyes@deyesinstance:~$ curl -V

curl 7.64.0 (x86_64-pc-linux-gnu) libcurl/7.64.0 OpenSSL/1.1.1d zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.5) libssh2/1.8.0 nghttp2/1.36.0 librtmp/2.3

Release-Date: 2019-02-06

Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp

Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

deyes@deyesinstance:~$

[Wayne Zhang]

BTW,  your cert is self signed,  you have to use "-k" with curl.  

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/fff17e90-7954-42d0-baac-51b606709be7n%40googlegroups.com.

[Ankit Dwivedi]

ok now the response is .

deyes@deyesinstance:~$ curl -k localhost:443/echo

curl: (52) Empty reply from server

or 

curl -k -d '{"message":"hello world"}' -H "content-type:application/json" https://localhost:443/echo?key=AIzaSyB1MV_bEbTsthB19clRLfjJHsSY8tIPMTI

curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to localhost:443

[Wayne Zhang]

add a --enable_debug flag to ESP arg,   and check Envoy debug log to see if there are any clues on the ssl negotiation. 

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/387d3bd3-9b9e-44ac-b2ec-e09a23442639n%40googlegroups.com.

[Wayne Zhang]

I just tested for localhost.  It works for me.   Here is the command I used

run ESP

sudo docker run     --detach     --name=esp     --publish=443:9000     --net=esp_net   --volume=/etc/esp/ssl:/etc/esp/ssl   gcr.io/endpoints-release/endpoints \
-runtime:2     --service=SERVICE_NAME     --rollout_strategy=managed     --listener_port=9000     --backend=http://echo:8080 --ssl_server_cert_path=/etc/esp/ssl \
--enable_debug

and curl

curl -k --request POST    --header "content-type:application/json"    --data '{"message":"hello world"}'    "https://localhost:443/echo?key=$KEY"
{
 "message": "hello world"
}

[qiwz...@google.com]

Any update?

[Ankit Dwivedi]

Thanks friend for your reply.

Issue is not resolved yet but we decided to use app engine now, and successfully able to deploy service on app engine but facing some challenges.

I am using firebase auth for my api and defined like below in openapi-appengine.yaml.

  security:

    - firebase: []

  securityDefinitions:

    firebase:

      authorizationUrl: ""

      flow: "implicit"

      type: "oauth2"

      # Replace YOUR-PROJECT-ID with your project ID

      x-google-issuer: "https://securetoken.google.com/deyes-295406"

      x-google-jwks_uri: "https://www.googleapis.com/robot/v1/metadata/x509/firebase-adminsdk-61fuh%40deyes-295406.iam.gserviceaccount.com"

      x-google-audiences: "deyes-295406"

and when using python client to hit api that details are given below- 

keyfile_jwt = generate_jwt("C:\\Users\\admin\\Downloads\\python-docs-samples-master\\appengine\\standard_python3\\hello_world\\pkey.json",

                               "https://securetoken.google.com/deyes-295406",

                               "deyes-295406",

                               3600)

    print(keyfile_jwt)

    make_jwt_request(keyfile_jwt, "https://deyes-dq5rvo4msq-el.a.run.app")

till to this point everything is running fine and I am able to get response when hitting my api.

But when I am checking in developer portal where my configuration is given below, facing issue.

api key - *************************

firebase auth domain - deyes-295406.firebaseapp.com

token Type - id token

error - {

  "message": "Jwks doesn't have key to match kid or alg from Jwt",

  "code": 401

}

token type - access token 

error -  {

  "code": 401,

  "message": "Jwt is not in the form of Header.Payload.Signature with two dots and 3 sections"

}

And after more research, we found that - 

when I am using - x-google-jwks_uri: "https://www.googleapis.com/service_accounts/v1/metadata/x509/firebase-ad...@testing-296402.iam.gserviceaccount.com"  in YAML configuration then I am able to hit api successfully from my python client successfully,

keyfile_jwt = generate_jwt("D:\\DEYES\\cloudApiEndPoints\\python-docs-samples-master\\appengine\\standard_python3\\hello_world\\fireAuth3.json",

                               "https://securetoken.google.com/testing-296402",

                               "firebase-ad...@testing-296402.iam.gserviceaccount.com",

                               "testing-296402",

                               expiry_length)

    print(keyfile_jwt)                               

    make_jwt_request(keyfile_jwt, "https://testing-deyes-tfk2y45mka-el.a.run.app/")

But falling when testing api from endpoint portal and getting error - 

{

  "message": "Jwks doesn't have key to match kid or alg from Jwt",

  "code": 401

}

And when I am using -  x-google-jwks_uri: "https://www.googleapis.com/service_accounts/v1/metadata/x509/secur...@system.gserviceaccount.com"  in YAML configuration then I am able to hit api successfully from my endpoint portal successfully,

But falling when testing using my python client with same above given details. and getting error -

{

  "message": "Jwks doesn't have key to match kid or alg from Jwt",

  "code": 401

}

We tried a lot to get some help from google on this but, unfortunately didn't get any reply on my query, finally I received your response, so please help me into this.
and one more thing- when I am trying to use auth0 authentication for my api - its asks me for the the  but I didn't find anything related to this in the documentation and same issue coming when using google token. and this issue is coming when I am testing my api from developer portal.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/e6a1e711-37f0-4ec5-8982-cca99e767d1an%40googlegroups.com.

[Wayne Zhang]

Endpoint developer portal could not work with other authentication other than api-key.    This is its limitation that is clearly documented here.  

-Wayne

[Ankit Dwivedi]

Thanks Wayne, for your kind support.

I will be thankful, if you will help me to clear one more doubt please,

if I am deploying my api backend on app engine and my service yaml file in cloud endpoint and now requesting api through cloud endpoint service url, in this case the flow of my api call will be - 

1. A request to your code is received and is sent to the Endpoints Management module within cloud Endpoints .
2. The Endpoints Management module sends a check request to Service Control.
3. If you configured your API to require an API key or authentication, Service Control checks to see if the request is permitted and sends a response back to the Endpoints Management module.
4. If the request isn't permitted, the Endpoints Management module rejects the request. If the request is permitted, it is forwarded to Endpoints Frameworks. Either way, the Endpoints Management module logs information about the request.
5. Endpoints Frameworks routes the request to your backend code.

if the above given steps are correct then in that case we are not using ESP/ESPV2 which we configured in step - 3 (https://cloud.google.com/endpoints/docs/openapi/get-started-app-engine-standard#reserve_hostname) in doc- 

Getting Started with Endpoints for App Engine standard environment with ESPv2

and esp is only useful if I will call my api through CLOUD_RUN_SERVICE_NAME  .

[Wayne Zhang]

You tried to describe a request data-path for Endpoint Frameworks.  But it is not correct. 

The big difference between Endpoint Framework and ESPv2 is:  Endpoint Framework is a library,  only in two languages; python and java.   If your backend application is in Python or Java,  you can call Endpoint framework library.   The code is running in the same machine as your backend code.  A request is received by the framework library,  and call remote ServiceControl if api-key is required,  then forwarded to your code.   Sorry, but Endpoint framework is deprecated now.  Here is the doc for endpoint framework

ESPv2 is a HTTP proxy.  All the Endpoint work is done in that proxy.  Your applications can be written in any language. You have to deploy ESPv2 proxy in front of your application, e.g. deployed in Cloud Run,  and your application deployed in AppEngine. A request is received by ESPv2 first,  it will call service control if api-ke is required, and forward the request to your backend. Here is the doc for ESPv2

[Mike DiChiappari]

If Endpoint Framworks is deprecated , why is it prominently featured in the Google documentation for Cloud Endpoints?

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/CAGJCt2OK7Uehtq_WpoCbT38top%3DqiyUR9ih_V4RbF6YSzq33fQ%40mail.gmail.com.

[Wayne Zhang]

Hi Mike,   we will remove the Endpoint framework document.  Thanks.  -Wayne

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/cbaa0e75-8d46-1c67-1f19-3de7daa5384e%40fittrace.com.

[Mike DiChiappari]

How long will EF be supported for?

[Wayne Zhang]

Hi Mike,   I need to defer to our PM to answer your question. 

[Ankit Dwivedi]

Hi Wayne,

Need help please, we are using service to service authentication in our implementation. I want to track uses of my service account which we are using for token creation. 

Can you please advise any way to track the uses of a service account.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/CAGJCt2PtW%3DndwBmZ-OVZczWNBZEq8ku3OVDfkVJNX_C-JVb8tw%40mail.gmail.com.

[Wayne Zhang]

Could you open a new thread for discussion?  Since this is a new issue so a new discussion thread is easier for others to search for.  Could you describe in detail how your service account is used and what information you want to keep track of