Securing Endpoints Frameworks with Google ID tokens

[que...@sensome.com]

Hello,

I am having some difficulty wrapping my head around the OAuth protocol used to secure a Cloud Endpoints API. Here's my situation:

• I am using Endpoints Frameworks (in Java) and configured it to require Google ID tokens.
  
• I have clients wishing to authenticate their requests to the Endpoints API. Clients are NodeJs apps running on tablets (ubuntu), so outside GCP obviously. 
  
• Since those clients are servers, there cannot be a user-consent step in the authentication protocol. I need each client to have their own service account. So I follow Google's implementation of OAuth 2.0 protocol, and use the service account private key stored in one those clients to create and sign a JWT. I exchange the JWT for a Google ID token, and attach that token to my Cloud Endpoints API requests.
  

I found this piece in Google's documentation (here): 

Important: We recommend that you use the Google ID JWT because this allows all of your service accounts to access the Google ID token server that you have configured in your OpenAPI configuration file. If instead you use a custom JWT signed only by the service account, you must explicitly list each service account that you wish to grant access to in the securityDefinitions section of your OpenAPI configuration file. This means that you'll have to redeploy the configuration whenever you want to grant access to a new service account.

This seemed to exactly fit my needs because I will have a lot of service accounts and do not want to redeploy my backend for each new client.

I implemented the solution on Configuring your API to support authentication (GOOGLE ID tab) on the server-side, and Using a Google ID token (KEY FILE tab) on the client-side. This works when using the EspAuthenticator. However, I quickly realized that any Google ID token obtained from anywhere could authenticate to my platform, and I am not sure I configured the Endpoints correctly. This may have to do with the Authenticator I chose.

From what I understand:

• GoogleOAuth2Authenticator: not suitable because Google ID tokens are apparently not OAuth2 tokens (checking the code, they do not have the prefix 'ya29.' or '1/' so the authenticator returns a User null every request)
  
  
• EspAuthenticator: This lets retrieve the User object, but does not seem to perform any check other than verifying the expiration date, the issuer (which is  https://acccounts.google.com) and the audiences. So anyone requesting a Google ID token from any service account can access my API (if he initially set the 'target_audience' field to API endpoints in the initial JWT). Moreover, Endpoints Frameworks does not use ESP but Google Service Control API.
  
  
• GoogleJWTAuthenticator: In the code of this class we can find the comment  "Authenticator for Google issued JSON Web Token, currently specific for Google Id Token" suggesting that its the right one to use. However I tested it and it requires listing every service account's email for the Google ID tokens to be accepted.

I am basically wondering what am I missing. The doc I quoted clearly says its possible to configure the service so that my service accounts will be granted access without explicitly listing them. So far I only managed to have these 2 configs: all the Google ID tokens ever created from everywhere, or all the Google ID tokens from service accounts explicitly listed.  

I know this is kind of a long post but I wanted to try to  be extra clear. Thanks for reading this far.

[Andrew Gunsch]

Hey, thanks for writing!

I think the crux of the issue is that Endpoints Frameworks and Endpoints with OpenAPI are implemented fairly differently, and you're looking at documentation that spans both. The details don't necessarily apply from one to the other.

* In Endpoints with OpenAPI, there's a proxy in front of your app (Endpoints Server Proxy). That proxy loads the OpenAPI config you pushed to Google (gcloud endpoints services deploy), which is why you can add service accounts to the OpenAPI config without redeploying your backend.

* In Endpoints Frameworks, there's no such proxy. The OpenAPI config is generated from your annotations, and is mostly so that Google can understand your API surface to be able to handle your API's metrics/logging/etc. correctly. But auth is handled by your app itself, which means you have to configure this in your app's auth annotations (see this doc) and redeploy when you make changes.

Hope that helps,

- Andrew

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/ab312b5e-9608-46ec-98bc-51d72aa17358%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Quentin Cavalié]

Thanks for answering so quickly, your response did clear things up, thanks a lot!

What I didn’t get before was that the role of the OpenAPI document was so different in the two contexts, and that I couldn’t expect the ESP OpenAPI document to work the same way in Frameworks.

Concerning the implementation of the auth for Endpoints Frameworks, what you’re saying is that there’s no way around listing explicitly each service account in the Annotations ? That redeploying the API each time is mandatory ?

Just out of curiosity, what if I was to accept all service accounts (simply by not listing any issuers), and then perform a regex check on the User email in my code ? Would that be enough to confirm that the issuer is from my project since the project ID is attached to the service account email and since each project ID is unique ?

In that scenario there wouldn’t be any need to redeploy the app as long as all the service accounts are from the same project. But apart from the fact that it’s a terribly ugly workaround, would that be enough to secure the API ? Would there be anyway for a malicious third party to manipulate a random Google ID token and trick Google’s Service Control API into thinking it’s from a service account from another project ?

Thanks again,

Quentin

Obtenez Outlook pour iOS

---

From: Andrew Gunsch <gun...@google.com>
Sent: Friday, August 10, 2018 8:36:58 PM
To: Quentin Cavalié
Cc: google-clou...@googlegroups.com
Subject: Re: Securing Endpoints Frameworks with Google ID tokens

 

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender by reply email and then delete this message and any attachments.

[Andrew Gunsch]

Inline:

On Friday, August 10, 2018 at 12:51:45 PM UTC-7, Quentin Cavalié wrote:

Thanks for answering so quickly, your response did clear things up, thanks a lot!

What I didn’t get before was that the role of the OpenAPI document was so different in the two contexts, and that I couldn’t expect the ESP OpenAPI document to work the same way in Frameworks.

Concerning the implementation of the auth for Endpoints Frameworks, what you’re saying is that there’s no way around listing explicitly each service account in the Annotations ? That redeploying the API each time is mandatory ?

Correct. Just a limitation of using the Frameworks rather than an ESP-based setup.

 

Just out of curiosity, what if I was to accept all service accounts (simply by not listing any issuers), and then perform a regex check on the User email in my code ? Would that be enough to confirm that the issuer is from my project since the project ID is attached to the service account email and since each project ID is unique ?

Interesting idea! I think that the framework will still validate the JWT, and then pass back the information in the User token, but I can't say for sure today whether that's true nor whether that's safe enough to fully secure the API. I'll see if we can get more information next week.

Obviously it would also depend on the correctness of your email validation and assumptions about what service account patterns you're accepting.

 

In that scenario there wouldn’t be any need to redeploy the app as long as all the service accounts are from the same project. But apart from the fact that it’s a terribly ugly workaround, would that be enough to secure the API ? Would there be anyway for a malicious third party to manipulate a random Google ID token and trick Google’s Service Control API into thinking it’s from a service account from another project ?

Thanks again,

Quentin

Obtenez Outlook pour iOS

---

From: Andrew Gunsch <gun...@google.com>
Sent: Friday, August 10, 2018 8:36:58 PM
To: Quentin Cavalié

Cc: google-cloud-endpoints@googlegroups.com

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endpoints+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/ab312b5e-9608-46ec-98bc-51d72aa17358%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jkr...@appirio.com]

Hi Andrew,

I am also very interested in this answer as I am comparing the returned user email from the Framework with a list of email addresses stored in Datastore to verify access. This way I can update datastore to allow additional service account's access to my API.

Regards,

James Krimm

On Fri, Aug 10, 2018 at 5:23 PM 'Andrew Gunsch' via Google Cloud Endpoints <google-clou...@googlegroups.com> wrote:

Inline:

On Friday, August 10, 2018 at 12:51:45 PM UTC-7, Quentin Cavalié wrote:

Thanks for answering so quickly, your response did clear things up, thanks a lot!

What I didn’t get before was that the role of the OpenAPI document was so different in the two contexts, and that I couldn’t expect the ESP OpenAPI document to work the same way in Frameworks.

Concerning the implementation of the auth for Endpoints Frameworks, what you’re saying is that there’s no way around listing explicitly each service account in the Annotations ? That redeploying the API each time is mandatory ?

Correct. Just a limitation of using the Frameworks rather than an ESP-based setup.

 

Just out of curiosity, what if I was to accept all service accounts (simply by not listing any issuers), and then perform a regex check on the User email in my code ? Would that be enough to confirm that the issuer is from my project since the project ID is attached to the service account email and since each project ID is unique ?

Interesting idea! I think that the framework will still validate the JWT, and then pass back the information in the User token, but I can't say for sure today whether that's true nor whether that's safe enough to fully secure the API. I'll see if we can get more information next week.

Obviously it would also depend on the correctness of your email validation and assumptions about what service account patterns you're accepting.

 

In that scenario there wouldn’t be any need to redeploy the app as long as all the service accounts are from the same project. But apart from the fact that it’s a terribly ugly workaround, would that be enough to secure the API ? Would there be anyway for a malicious third party to manipulate a random Google ID token and trick Google’s Service Control API into thinking it’s from a service account from another project ?

Thanks again,

Quentin

Obtenez Outlook pour iOS

---

From: Andrew Gunsch <gun...@google.com>
Sent: Friday, August 10, 2018 8:36:58 PM
To: Quentin Cavalié

Cc: google-clou...@googlegroups.com

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/ab312b5e-9608-46ec-98bc-51d72aa17358%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender by reply email and then delete this message and any attachments.

--

You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/b9412e65-8fdb-4e2f-b264-9e088210e5cc%40googlegroups.com.

[D. T.]

The authenticators don't currently support arbitrary issuers, but you could implement your own Authenticator class and put that in your annotation, since you're using Java. Since all of the authenticators we have are open source, you can modify them to your needs.

Daniel Tang |

Software Engineer |

ta...@google.com |

Google LLC

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/CANMRvhnyyGE9e4QZiM6x30B6H9GjsKbTOyjviwi2G2gg0y5AAA%40mail.gmail.com.

[jkr...@appirio.com]

Hi Daniel,

What if instead of using service account auth you use Google ID Token? From my understanding this allows any Google account (user or service account) to authorize to the API and then you can write your code to allow or deny based on the user email you get from the Framework.

If I understand Quentin's question correctly, he and I both want to know are we safe assuming the email address provided by calling 

Python: endpoints.get_current_user().email() 

Java: user.getEmail()

is verified by Google and can not be faked?

At the bottom of this page from Google's documentation, for python which I am using.

https://cloud.google.com/endpoints/docs/frameworks/python/authenticating-users#authenticating_with_a_service_account 

For Java: 

https://cloud.google.com/endpoints/docs/frameworks/java/authenticating-users#authenticating_with_a_service_account

Regards,

James Krimm

See My Availability

Office: (317) 759-4943 | Cell: (859) 339-9493  | E-mail: jkr...@appirio.com

Cloud Powered Business Blog: blog.appirio.com | twitter: twitter.com/appirio

www.appirio.com

Accelerating Enterprise Adoption of the Cloud

[que...@sensome.com]

Hi Daniel, 

James is right in saying that my question was about the email retrieved by the EspAuthenticator, and more specifically if we can really expect the protocol to send back trustworthy user informations.

The authenticators don't currently support arbitrary issuers

I am not sure what you mean by that, but after looking around a bit more it seems to me that user emails and user ids cannot be faked, and that once its passes the EspAuthenticator checks we can implement our own validation process based the user email or id. (regex check, database lookup, IAM API call to check if it belongs to a known and authorized project, etc.)

I'd like however to have confirmation on this because I am not very knowledgable in security. To me the protocol is as follow:

• Create and sign a JWT with the service account private key.
  
  
• Send the JWT to Google's OAuth 2.0 authorization service for a Google ID token.
  
• The authorization service checks the JWT with the service account public key held by Google. This should mean that a random service account cannot manipulate the fields present in its private key to try to impersonate another account.
  
  
• Send the Google ID token to the properly configured Cloud Endpoints Frameworks API.
  
  
• The EspAuthenticator picks up the Google ID token and then performs these checks:
• validates the Google ID token claims (audiences, issuers, experiration date, etc.)
• checks the signature of the token.
  

I am not very sure about what's the mechanism behind the signature check of the token and could not find much information on it anywhere. I guess it might be another public/private key verification where both keys are kept by Google (public key being available at: https://www.googleapis.com/oauth2/v1/certs). If I try sending a JWT signed with a service account (with fake claims e.g. issuer='https://accounts.google.com' ) directly to my Endpoints API that did not initially go through Google's authorization server, this is the response I get  :

2018-08-14 13:27:43.609 CESTcom.google.api.server.spi.auth.EspAuthenticator authenticate: Authentication failed: com.google.common.util.concurrent.UncheckedExecutionException: com.google.api.auth.UnauthenticatedException: Failed to verify the signature of the auth token (EspAuthenticator.java:86)

After messing around with the JWTs, the private keys of my service accounts, and the config of my Endpoints Frameworks API I think I understand better how all this works together. I think the email and ids retrieved by the EspAuthenticator can be trusted and therefore another check can be performed at the API level so that everyone can implement their own authorization rules tailored to their needs. This avoids having to implement a custom Authenticator class and having to maintain that component as well, and this avoids having to list all the service accounts in your @Api annotation.

I guess I am trying to get confirmation on all of this. Have I missed something that would make trusting the user email or id impossible ? Once again I don't know much about security.

This the annotations used in my Cloud Endpoints API:

@Api(name = "my-api",
     version = "v1",
     title = "My API",
     authenticators = {EspAuthenticator.class},
     namespace = @ApiNamespace(ownerDomain = "my-api.com",
                               ownerName = "my-api.com"),
        audiences = "https://my-project-id.appspot.com")

This is the OpenAPI document generated with the maven endpoints frameworks plugin:

{
 "swagger": "2.0",
 "info": {
  "version": "1.0.0",
  "title": "my-project-id.appspot.com"
 },
 "host": "my-project-id.appspot.com",
 "basePath": "/",
 "schemes": [
  "https"
 ],
 "consumes": [
  "application/json"
 ],
 "produces": [
  "application/json"
 ],
 "paths": {
  "/my-api/v1/my-method": {

    ...

    "security": [
     {
      "google_id_token": [ ]
     },
     {
      "google_id_token_https": [ ]
     }
    ],
    "x-security": [
     {
      "google_id_token": {
       "audiences": [
        "https://my-project-id.appspot.com"
       ]
      }
     },
     {
      "google_id_token_https": {
       "audiences": [
        "https://my-project-id.appspot.com"
       ]
      }
     }
    ]
   }
  }
 },
 "securityDefinitions": {
  "google_id_token_https": {
   "type": "oauth2",
   "authorizationUrl": "",
   "flow": "implicit",
   "x-google-issuer": "https://accounts.google.com",
   "x-google-jwks_uri": "https://www.googleapis.com/oauth2/v1/certs"
  },
  "google_id_token": {
   "type": "oauth2",
   "authorizationUrl": "",
   "flow": "implicit",
   "x-google-issuer": "accounts.google.com",
   "x-google-jwks_uri": "https://www.googleapis.com/oauth2/v1/certs"
  }
 },
 "definitions": {
  ...

 }
}

[Andrew Gunsch]

That all looks reasonable to me --- though in a "IANAL" sense, I Am Not A Security Engineer, so take it with a grain of salt, but my reading of the code is that the crux is here:

* The JWT token is validated

* The issuer in the JWT token needs to be an expected issuer from your OpenAPI spec

* The UserInfo that's built uses the "email" field from the JWT claimset

Based on that reading, this seems like a reasonable approach for checking against N users.

If you want to look at how the JWT is verified, you can dig into:
* DefaultAuthTokenVerifier, which loads the key from

* DefaultJwksSupplier, which has determined the correct config from

* DefaultKeyUriSupplier, which (in the Google case) loads the config pointing to the public key from

* This config URL, pointing to this public key

This is for my reference as much as yours --- I learned all this digging through the code just now :)

--
You received this message because you are subscribed to the Google Groups "Google Cloud Endpoints" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-endp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-endpoints/2b15d0f9-c1a8-4b62-94df-534368976cb0%40googlegroups.com.

[que...@sensome.com]

Thanks for all the help Andrew, and thanks for the pointers. This has been of great help.