Problem authenticating to GAE app using GoogleCredential OAuth2

319 views
Skip to first unread message

Julian Bunn

unread,
Aug 5, 2015, 1:32:41 AM8/5/15
to Google App Engine

I have a GAE application with an endpoint that requires authentication, which I need to call from an application (rather than from in a browser). I was using ClientLogin, but that is now obsolete, so I have set up a Service Account in the Google Console, and stored its keypair .p12 file so that I can use the OAuth methods as described in the documentation.

Although the GoogleCredential builder successfully returns an authorization token, if I then use that token in an HTTP get call to the endpoint, the response is always the Google Login page.

Why, if I use the token, does GAE not take my application call as authorized? Am I doing this all wrong or missing a step? 

Here is the code:

    String emailAddress = "XXXX...@developer.gserviceaccount.com";
    JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
    String emailScope = "https://www.googleapis.com/auth/userinfo.email";
    String keyFileName = "YYYYY.p12";
    String baseURL = "http://ZZZZZ.appspot.com";
    HttpTransport httpTransport;
    try {
        httpTransport = GoogleNetHttpTransport.newTrustedTransport();

        File keyFile = new File(keyFileName);
        if(!keyFile.exists()) {
            System.err.println("Key file "+keyFileName+" missing");
            System.exit(0);
        }

        GoogleCredential credential = new GoogleCredential.Builder()
        .setTransport(httpTransport)
        .setJsonFactory(JSON_FACTORY)
        .setServiceAccountId(emailAddress)
        .setServiceAccountScopes(Collections.singleton(emailScope))
        .setServiceAccountPrivateKeyFromP12File(keyFile)
        .build();

        boolean success = credential.refreshToken();
        System.out.println("Access token refresh "+ success);

        String token = credential.getAccessToken();

        System.out.println("Token "+token);

        String uri = "http://ZZZZZ.appspot.com/gcm/home";

        System.out.println("uri: " + uri);

        HttpGet get = new HttpGet(uri);
        get.setHeader("Cookie", token);

        HttpClient client = new DefaultHttpClient();
        HttpResponse response = client.execute(get);
        response.getEntity().writeTo(System.out);

Typical output:

   Access token refresh true
   Token ya29.xQGG1kxxxxxxxxxxxxxxxxxxx
   uri: http://ZZZZZ.appspot.com/gcm/home

   <!DOCTYPE html>
   <html lang="en">
      <head>
      <meta charset="utf-8">
      <meta content="width=300, initial-scale=1" name="viewport">
      <meta name="google-site-verification" content="LrdTUW9psUAMbh4Ia074-BPEVmcpBxF6Gwf0MSgQXZs">
      <title>Sign in - Google Accounts</title>
      .....

Nick (Cloud Platform Support)

unread,
Aug 5, 2015, 7:57:26 PM8/5/15
to Google App Engine
Hi Julian,

You've produced an excellent post which would belong on stackoverflow.com. Google Groups isn't the place to post specific technical issues, as this forum is meant more for general discussion of the platform and services. 

I'll give you the advice before you post there that it seems you've combined examples from different kinds of OAuth flow and this might be the cause of your issues. I see that there's a variable "emailScope" - this is a scope which a user would actually grant to your application, not one which a service account could grant.

The service account and its credentials are used to call APIs on behalf of your application, although I don't think I've seen this pattern before, where you want to call an endpoint on your own app using a service account. As far as I know, service accounts have only been used to authenticate with Google APIs, although I suppose it might be possible to write an endpoint which correctly authenticates it.

You could do some more reading on OAuth2, OpenID Connect, Service Accounts, and the Google Identity Platform, and try to repost your question to stackoverflow.com. That would be the best action as there are many more users there ready to help with a technical question.

If you would like to open a thread in this forum discussing the platform or services in more broad terms, starting a discussion that would be useful for other users to join in to, feel free to do so.

Have a great day!


On Wednesday, August 5, 2015 at 1:32:41 AM UTC-4, Julian Bunn wrote:

I have a GAE application with an endpoint that requires authentication, which I need to call from an application (rather than from in a browser). I was using ClientLogin, but that is now obsolete, so I have set up a Service Account in the Google Console, and stored its keypair .p12 file so that I can use the OAuth methods as described in the documentation.

Although the GoogleCredential builder successfully returns an authorization token, if I then use that token in an HTTP get call to the endpoint, the response is always the Google Login page.

Why, if I use the token, does GAE not take my application call as authorized? Am I doing this all wrong or missing a step? 

Here is the code:

    String emailAddress = "XXXXXXXX@developer.gserviceaccount.com";
    JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
    String emailScope = "https://www.googleapis.com/auth/userinfo.email";
    String keyFileName = "YYYYY.p12";
    String baseURL = "http://ZZZZZ.appspot.com";
    HttpTransport httpTransport;
    try {
        httpTransport = GoogleNetHttpTransport.newTrustedTransport();

        File keyFile = new File(keyFileName);
        if(!keyFile.exists()) {
            System.err.println("Key file "+keyFileName+" missing");
            System.exit(0);
        }

        GoogleCredential credential = new GoogleCredential.Builder()
        .setTransport(httpTransport)
        .setJsonFactory(JSON_FACTORY)
        .setServiceAccountId(emailAddress)
        .setServiceAccountScopes(Collections.singleton(emailScope))
        .setServiceAccountPrivateKeyFromP12File(keyFile)
        .build();

        boolean success = credential.refreshToken();
        System.out.println("Access token refresh "+ success);

        String token = credential.getAccessToken();

        System.out.println("Token "+token);

        String uri = "http://ZZZZZ.appspot.com/gcm/home";

        System.out.println("uri: " + uri);

        HttpGet get = new HttpGet(uri);
        get.setHeader("Cookie", token);

        HttpClient client = new DefaultHttpClient();
        HttpResponse response = client.execute(get);
        response.getEntity().writeTo(System.out);

Julian Bunn

unread,
Aug 6, 2015, 3:20:28 PM8/6/15
to Google App Engine
Hi Nick,

Many thanks - I had already posted on stackoverflow with no luck, so came here :-) I do have one reply now over there, which suggests using client secrets, so that is a good lead. Also your comments on the use of service account are well taken - it looks like that may be inappropriate.

Thanks for the pointers to the documentation, which I'd already visited and read but ended up being confused - as is no doubt evident from my question :-)

Julian


On Wednesday, August 5, 2015 at 4:57:26 PM UTC-7, Nick (Cloud Platform Support) wrote:
Hi Julian,

You've produced an excellent post which would belong on stackoverflow.com. Google Groups isn't the place to post specific technical issues, as this forum is meant more for general discussion of the platform and services. 

I'll give you the advice before you post there that it seems you've combined examples from different kinds of OAuth flow and this might be the cause of your issues. I see that there's a variable "emailScope" - this is a scope which a user would actually grant to your application, not one which a service account could grant.

The service account and its credentials are used to call APIs on behalf of your application, although I don't think I've seen this pattern before, where you want to call an endpoint on your own app using a service account. As far as I know, service accounts have only been used to authenticate with Google APIs, although I suppose it might be possible to write an endpoint which correctly authenticates it.

You could do some more reading on OAuth2, OpenID Connect, Service Accounts, and the Google Identity Platform, and try to repost your question to stackoverflow.com. That would be the best action as there are many more users there ready to help with a technical question.

If you would like to open a thread in this forum discussing the platform or services in more broad terms, starting a discussion that would be useful for other users to join in to, feel free to do so.

Have a great day!


On Wednesday, August 5, 2015 at 1:32:41 AM UTC-4, Julian Bunn wrote:

I have a GAE application with an endpoint that requires authentication, which I need to call from an application (rather than from in a browser). I was using ClientLogin, but that is now obsolete, so I have set up a Service Account in the Google Console, and stored its keypair .p12 file so that I can use the OAuth methods as described in the documentation.

Although the GoogleCredential builder successfully returns an authorization token, if I then use that token in an HTTP get call to the endpoint, the response is always the Google Login page.

Why, if I use the token, does GAE not take my application call as authorized? Am I doing this all wrong or missing a step? 

Here is the code:

    String emailAddress = "XXXX...@developer.gserviceaccount.com";
    JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
    String emailScope = "https://www.googleapis.com/auth/userinfo.email";
    String keyFileName = "YYYYY.p12";
    String baseURL = "http://ZZZZZ.appspot.com";
    HttpTransport httpTransport;
    try {
        httpTransport = GoogleNetHttpTransport.newTrustedTransport();

        File keyFile = new File(keyFileName);
        if(!keyFile.exists()) {
            System.err.println("Key file "+keyFileName+" missing");
            System.exit(0);
        }

        GoogleCredential credential = new GoogleCredential.Builder()
        .setTransport(httpTransport)
        .setJsonFactory(JSON_FACTORY)
        .setServiceAccountId(emailAddress)
        .setServiceAccountScopes(Collections.singleton(emailScope))
        .setServiceAccountPrivateKeyFromP12File(keyFile)
        .build();

        boolean success = credential.refreshToken();
        System.out.println("Access token refresh "+ success);

        String token = credential.getAccessToken();

        System.out.println("Token "+token);

        String uri = "http://ZZZZZ.appspot.com/gcm/home";

        System.out.println("uri: " + uri);

        HttpGet get = new HttpGet(uri);
        get.setHeader("Cookie", token);

        HttpClient client = new DefaultHttpClient();
        HttpResponse response = client.execute(get);
        response.getEntity().writeTo(System.out);

Nick (Cloud Platform Support)

unread,
Aug 6, 2015, 7:28:25 PM8/6/15
to Google App Engine
Hey Julian,

Glad to hear you're getting some help over there and my comment was helpful. If you post the link, I'll be glad to continue watching that thread and see if I can find anything else. 

In general, as said, it's not the right forum for specific issues, but either way you're here so I figured it wouldn't hurt to provide some advice while redirecting you. 

Also, as said, feel free to open threads here of more general discussion about the platform and services, if you'd like.

Best wishes,

Nick

Jason Collins

unread,
Aug 8, 2015, 12:38:00 PM8/8/15
to Google App Engine
Julian, can you post your link to your SO question?

Julian Bunn

unread,
Aug 8, 2015, 1:51:24 PM8/8/15
to Google App Engine
Hi Jason,


The suggestion there involves the Google Drive API, which is not really helping me, as my GAE application does not use that API.

Julian

Nick (Cloud Platform Support)

unread,
Aug 10, 2015, 7:13:26 PM8/10/15
to Google App Engine
Hi Julian,

The example code given there might be dealing with the Drive API, but APIs in this context are quite abstract, and you can easily substitute any Google API. 

Reading back over your question, I'm not sure you've supplied enough information for anybody to help answer. What exactly is doing the authenticating? Is your endpoint a Cloud Endpoints endpoint? It's not really clear to me what is doing the authentication at your "endpoint". Do you just mean that you've deployed with "login: admin"?

At any rate, this forum, as mentioned, isn't meant for 1-on-1 technical support, so I don't think you should continue to follow-up in this thread, and should either improve the stackoverflow question to clarify exactly what you're expecting to happen in technical language and specifics, or else post a new question which does include that information. That will enable people to help you better.

Best wishes,

Nick

Julian Bunn

unread,
Aug 10, 2015, 8:10:35 PM8/10/15
to google-a...@googlegroups.com
Hi Nick,

Thanks ... 

GAE is doing the authentication. My GAE app has endpoints (i.e. urls like my.appspot.com/gcm/home) that can only be executed by an admin who is logged in. There is nothing special I have implemented to support this, I am just using Google's GAE infrastructure.

So, in the past, all I needed to do from a client application was to call ClientLogin with a user/pass pair, which would return me a token which could then be sent as a Cookie in calls to the GAE endpoints.

This worked very well! 

Now that ClientLogin has been disabled, I am looking for an alternative to it. I apparently need to use OAuth2, but there is no documentation that seems to match my use case, unhappily. Use cases seem to assume the use of various Google APIs, which I am not using.

Thanks anyway.

Julian



--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/rYKSUDTFWTI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/a1c82338-d11c-4192-b020-d7f3107e87f9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Nick (Cloud Platform Support)

unread,
Aug 12, 2015, 4:28:20 PM8/12/15
to Google App Engine
Hi Julian,

OAuth2 is a complex topic and has many methods of application, being just an authentication/authorization protocol, and having many possible uses / forms of appearance (client-server, server-server, 3-legged, etc.)

From your comments, I can now understand you're using login: admin on a route of your app, and you'd like to know how to make requests to a route on your app protected in such a manner, using a service account to login. Is that accurate?

Could you let me know whether the service account is added as an admin of your application in the Developers Console under "Credentials" and whether your app's authentication method is set to "Google Accounts"?
To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.

Julian Bunn

unread,
Aug 12, 2015, 5:00:53 PM8/12/15
to Google App Engine
Hi Nick,

Thanks for much for persisting with your help!

Yes. your understanding is correct: I'd like to use a service account to login so that I can make requests to an admin route of my app.

Under "Credentials" in the new Google Developers Console, I see the Service Account listed in the OAuth section, with its ID, email address and certificate fingerprints.

Under "Permissions" in the Console, I have my own account and a maintenance account listed as Owners. On the same page, under "Service Accounts" I have three listed, all having Edit permission. One of these is the same account listed in "Credentials". (The other two are @cloudservices and @developer.gservice accounts - I don't know where they came from, as I don't recall creating them).

On the old version of the Developers Console, I can see that the Authentication Type is set to Google Accounts API. On there I can also see the Service Account Name but it is different from the Service Account listed under Credentials (above) - which is confusing me.

The web xml for the deployment includes:

<!-- Secure sensitive URLs -->
<security-constraint>
<web-resource-collection>
<url-pattern>/gcm/home</url-pattern>
<url-pattern>/gcm/send</url-pattern>
</web-resource-collection>
<auth-constraint>
<role-name>admin</role-name>
</auth-constraint>
</security-constraint>

These are the two endpoints I need to call from the client.

Thanks again!
Julian

To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.

Nick (Cloud Platform Support)

unread,
Aug 13, 2015, 7:39:08 PM8/13/15
to Google App Engine
Hi Julian,

Thank you for your detailed response. I'm looking into what you've posted and attempting to work through an implementation of what you describe, or at least attempting to determine if it's possible. I'll let you know shortly any findings or solutions I can recommend.

Best wishes,

Nick (Cloud Platform Support)

unread,
Aug 14, 2015, 5:03:12 PM8/14/15
to Google App Engine
A quick question, is it possible you could provide the skeleton code for your client project? It appears to be a standalone java program, rather than a web app, yes?

Julian Bunn

unread,
Aug 14, 2015, 5:44:08 PM8/14/15
to google-a...@googlegroups.com
Hi Nick,

Yes ... here are the relevant details (extracted from my updated question on stack overflow). First the secure URL specs:

<!-- Secure sensitive URLs -->
<security-constraint>
    <web-resource-collection>
        <url-pattern>/gcm/home</url-pattern>
        <url-pattern>/gcm/send</url-pattern>
    </web-resource-collection>
    <auth-constraint>
        <role-name>admin</role-name>
    </auth-constraint>
</security-constraint>

Previously, in the client application, I was using ClientLogin to authenticate with Google before calling the endpoint. This is the code I was using, that extracts the "Auth" token which it then uses as a Cookie on HTTP GET to the above endpoints.

public static String loginToGoogle(String userid, String password,        
        String appUrl) throws Exception {        
    HttpClient client = new DefaultHttpClient();        
    HttpPost post = new HttpPost(        
            "https://www.google.com/accounts/ClientLogin");        

    MultipartEntity reqEntity = new MultipartEntity();        
    reqEntity.addPart("accountType", new StringBody("HOSTED_OR_GOOGLE",        
            "text/plain", Charset.forName("UTF-8")));        
    reqEntity.addPart("Email", new StringBody(userid));        
    reqEntity.addPart("Passwd", new StringBody(password));        
    reqEntity.addPart("service", new StringBody("ah"));        
    reqEntity.addPart("source", new StringBody("WWWWmyappname"));        
    post.setEntity(reqEntity);        
    HttpResponse response = client.execute(post);        
    if (response.getStatusLine().getStatusCode() == 200) {        
        InputStream input = response.getEntity().getContent();        
        String result = IOUtils.toString(input);        
        String authToken = getAuthToken(result);        
        post = new HttpPost(appUrl + "/_ah/login?auth=" + authToken);        
        response = client.execute(post);        
        Header[] cookies = response.getHeaders("SET-COOKIE");        
        for (Header cookie : cookies) {        
            if (cookie.getValue().startsWith("ACSID=")) {        
                return cookie.getValue();        
            }        
        }        
        throw new Exception("ACSID cookie cannot be found");        
    } else        
        throw new Exception("Error obtaining ACSID");        
}        

private static String getAuthToken(String responseText) throws Exception {        
    LineNumberReader reader = new LineNumberReader(new StringReader(        
            responseText));        
    String line = reader.readLine();        
    while (line != null) {        
        line = line.trim();        
        if (line.startsWith("Auth=")) {        
            return line.substring(5);        
        }        
        line = reader.readLine();        
    }        
    throw new Exception("Could not find Auth token");        
}

​Calling the gcm endpoint:

HttpGet get = new HttpGet(httpURL);

get.setHeader("Cookie", authCookie);


HttpResponse response = client.execute(get);

response.getEntity().writeTo(System.out);
​
​where "authCookie" is the token obtained from loginToGoogle above.

Thanks so much for helping with this!

Julian​

Nick (Cloud Platform Support)

unread,
Aug 18, 2015, 7:30:08 PM8/18/15
to Google App Engine
Hi Julian,

So, after some extensive testing and reading around online, it appears that this is no longer possible, and I encourage you to make a public issue tracker feature request explaining in simple terms, minus the code, your desired use-case, of making HTTP calls to your app on routes protected by login: admin.

From what I can see from looking around online, the old method which used a certain ClientLogin endpoint to get the token passed to the server as the ACSID cookie is no longer active. The closest thing to signing-in with a service account that I could find was service account Apps domain user impersonation in the Server to Server OAuth2 docs. After extensive testing, implementing code which built the credential, built an HttpTransport for it, it was not possible to get the app to recognize the calling java code as a "login: admin" user, even when impersonating a user which has admin status (standalone java code compiled with classpath by hand).

So, feel free to post the public issue tracker feature request link in this thread once you've made it, and I'll be following it and helping to get it processed.

Best wishes,

Nick
To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.

Julian Bunn

unread,
Aug 18, 2015, 7:44:44 PM8/18/15
to google-a...@googlegroups.com
Hi Nick,

Thanks very much for investigating this so thoroughly and following up. I will complete a feature request, as you suggest, probably sometime tomorrow.

It's reassuring that you too were unable to make it work - I'd tried so many different things myself, I was beginning to doubt my competence :-)

Julian

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/rYKSUDTFWTI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/bb96ccff-5870-45ea-a4ac-9405bc16f738%40googlegroups.com.

Julian Bunn

unread,
Aug 19, 2015, 1:57:21 PM8/19/15
to google-a...@googlegroups.com
Issue 12272: https://code.google.com/p/googleappengine/issues/detail?id=12272

Let me know if it needs any elaboration.

Thanks again,
Julian

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/rYKSUDTFWTI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/bb96ccff-5870-45ea-a4ac-9405bc16f738%40googlegroups.com.

Nick (Cloud Platform Support)

unread,
Aug 19, 2015, 2:59:25 PM8/19/15
to Google App Engine
Hi Julian,

That looks good, and no worries, for my part I'm glad you've been so clear and cooperative throughout this. I'm currently taking a look at the issue and forwarding it to those concerned, with a link to this thread. I'll update the issue tracker thread shortly.

Regards,

Nick
To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/rYKSUDTFWTI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages