Webservice authentication (Python)

[epb]

Hi,

I am trying to make my Python program talk to a web service hosted on
app engine. I am using the appengine-rest-server and authentication
using a Google account is required on the server. The idea is that the
user specifies his username/password in the client application, and
then the client app. will talk to app engine server via webservices.

I found this post:
http://groups.google.com/group/google-appengine/browse_thread/thread/3bd69f0aa72d4bcb/8f872170bd996ad6?lnk=gst&q=webservice+authentication#8f872170bd996ad6.
The guy who replied proposes the use of Authentication for Installed
Apps, but also mentions that the GAE service should be "ah", which it
is not according to http://code.google.com/apis/base/faq_gdata.html#clientlogin
(in fact, GAE is not even listed there). What is the service name for
GAE (if it exists) and can I access the GAE application in this way?
Given the service name I should be able to talk to the server like
this:

import urllib, urllib2

# user-data
data = urllib.urlencode({'accountType' : 'GOOGLE', 'Email' :
'x...@gmail.com', 'Passwd': 'xxx', 'service': 'someservice' ,
'source' : 'Me-MyApp-1.0'})

# do the log-in
f = urllib.urlopen('https://www.google.com/accounts/ClientLogin',data)

# read ClientLogin token (ugly)
line = f.readline()
token = ''
while line:
if line.startswith('Auth'):
token = line[5:]
line = g.readline()

# if we got token, login was completed
if token:
# URL to a Model called testmodel (via appengine-rest-server)
url = 'http://myappengineapp.appspot.com/rest/testmodel'
# add Authorization header with token
headers = {'Authorization': 'GoogleLogin auth='+token}
handler = urllib2.HTTPHandler()
opener = urllib2.build_opener(handler)
req = urllib2.Request(url, headers=headers)
f = opener.open(req)

# Do something with response....

# close "files"
f.close()
g.close()

Correct? In fact, I am able to login, but somehow I can't use the
token to access the GAE app..

[Nick Johnson (Google)]

On Tue, Jul 14, 2009 at 8:03 PM, epb<esben...@gmail.com> wrote:
>
> Hi,
>
> I am trying to make my Python program talk to a web service hosted on
> app engine. I am using the appengine-rest-server and authentication
> using a Google account is required on the server. The idea is that the
> user specifies his username/password in the client application, and
> then the client app. will talk to app engine server via webservices.

You may want to look at appengine_rpc.py, in the SDK
google.appengine.tools package. It is designed specifically for this.

>
> I found this post:
> http://groups.google.com/group/google-appengine/browse_thread/thread/3bd69f0aa72d4bcb/8f872170bd996ad6?lnk=gst&q=webservice+authentication#8f872170bd996ad6.
> The guy who replied proposes the use of Authentication for Installed
> Apps, but also mentions that the GAE service should be "ah", which it
> is not according to http://code.google.com/apis/base/faq_gdata.html#clientlogin
> (in fact, GAE is not even listed there). What is the service name for
> GAE (if it exists) and can I access the GAE application in this way?

The service is, as mentioned, 'ah', and yes, you can authenticate
using this method.

> Given the service name I should be able to talk to the server like
> this:
>
> import urllib, urllib2
>
> # user-data
> data = urllib.urlencode({'accountType' : 'GOOGLE', 'Email' :
> 'x...@gmail.com', 'Passwd': 'xxx', 'service': 'someservice' ,
> 'source' : 'Me-MyApp-1.0'})
>
> # do the log-in
> f = urllib.urlopen('https://www.google.com/accounts/ClientLogin',data)
>
> # read ClientLogin token (ugly)
> line = f.readline()
> token = ''
> while line:
>    if line.startswith('Auth'):
>        token = line[5:]
>    line = g.readline()

The returned body is urlencoded, so you can use the built in
functionality for this.

>
> # if we got token, login was completed
> if token:
>    # URL to a Model called testmodel (via appengine-rest-server)
>    url = 'http://myappengineapp.appspot.com/rest/testmodel'
>    # add Authorization header with token
>    headers = {'Authorization': 'GoogleLogin auth='+token}
>    handler = urllib2.HTTPHandler()
>    opener = urllib2.build_opener(handler)
>    req = urllib2.Request(url, headers=headers)
>    f = opener.open(req)

App Engine apps do not accept authorization headers - instead, you
need to make a request to a special URL with the token to get a user
cookie back. See appengine_rpc.py for details.

-Nick Johnson

>
>    # Do something with response....
>
>    # close "files"
>    f.close()
> g.close()
>
> Correct? In fact, I am able to login, but somehow I can't use the
> token to access the GAE app..
>
> >
>

--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number: 368047

[Tony Rowles]

Since I happened to have this up, here's a bit of sample code to get
an authentication cookie for an appspot app...

from google.appengine.api import urlfetch
from urllib import urlencode
email = request.POST['username']
passwd = request.POST['password']
serv_root = "http://myapp.appspot.com"
target = 'http://myapp.appspot.com/null'
app_name = "myapp-1.0"
auth_uri = 'https://www.google.com/accounts/ClientLogin'
authreq_data = urlencode({ "Email": email,
"Passwd": passwd,
"service": "ah",
"source": app_name,
"accountType": "HOSTED_OR_GOOGLE" })
result = urlfetch.fetch(auth_uri, authreq_data, method=urlfetch.POST,
follow_redirects=False)
auth_dict = dict(x.split("=") for x in result.content.split("\n") if
x)
auth_token = auth_dict["Auth"]
serv_args = {}
serv_args['continue'] = target
serv_args['auth'] = auth_token
serv_uri = "%s/_ah/login?%s" % (serv_root, urlencode(serv_args))
result2 = urlfetch.fetch(serv_uri, follow_redirects=False,
method=urlfetch.GET)
### here's the cookie which will authenticate future requests
cookie = result2.headers['set-cookie'].split(';')[0]
# cookie[0] => "ACSID"
# cookie[1] => "AAAAHFSDJHSDFHSDJFHSDJFHSJFSDfsdjfhsjdfhsjdfh..."

[epb]

Thanks for your answers.

As I understand Nick's response, I only need to use appengine_rpc.py
for the entire process. I tried the following:

-------

def passwdFunc():
return ('my_email','my_passwd')

rpcServer = appengine_rpc.HttpRpcServer
('myapp.appspot.com',passwdFunc,None,'myAppName')
blah = rpcServer.Send('/')

-------

This gave me a 302 error and the following log:

-------

Server: myapp.appspot.com
Sending HTTP request:
POST /? HTTP/1.1
Host: myapp.appspot.com
X-appcfg-api-version: 1
Content-type: application/octet-stream


Got http error, this is try #1
Got 302 redirect. Location:
https://www.google.com/accounts/ServiceLogin?service=ah&continue=http://myapp.appspot.com/_ah/login%3Fconti
nue%3Dhttp://myapp.appspot.com/
&ltmpl=gm&ahname=MyAppName&sig=46378246....321321312
Sending HTTP request:
POST /? HTTP/1.1
Host: myapp.appspot.com
X-appcfg-api-version: 1
Content-type: application/octet-stream


Got http error, this is try #2

-------

It seems to me that the Send() function should do all authentication-
work automatically and re-direct to the app page after logging in.
Right?

Anyway, I'll try out Tonys solution also..

[epb]

I can see why Tony's version would work. His "algorithm" has two
steps:

1. Get the authorization token using ClientLogin (which I also managed
to do).
2. Use the uri "servername/_ah/login" to get the auth. cookie.

The appengine_rpc module seems to do authentication in a similar way:

A. Try to access the app. This results in a redirect to a location
that starts with https://www.google.com/accounts/ServiceLogin
B. Get a auth. token (like step 1 above)
C. Use auth. token to get auth. cookie.
D. Try to access the app. again (this is where it fails in my case...)

Anyway, step C is performed using the function below:

-------

def _GetAuthCookie(self, auth_token):
"""Fetches authentication cookies for an authentication token.

Args:
auth_token: The authentication token returned by ClientLogin.

Raises:
HTTPError: If there was an error fetching the authentication
cookies.
"""
continue_location = "http://localhost/"
args = {"continue": continue_location, "auth": auth_token}
login_path = os.environ.get("APPCFG_LOGIN_PATH", "/_ah")
req = self._CreateRequest("%s://%s%s/login?%s" %
(self.scheme, self.host, login_path,
urllib.urlencode(args)))
try:
response = self.opener.open(req)
except urllib2.HTTPError, e:
response = e
if (response.code != 302 or
response.info()["location"] != continue_location):
raise urllib2.HTTPError(req.get_full_url(), response.code,
response.msg,
response.headers, response.fp)
self.authenticated = True

------

It seems to me, that we do nothing with the response in this
function?? Shouldn't we save the cookie in the response like Tony's
does above, and then use it when we try to log in again?

On Jul 15, 1:06 pm, epb <esbenbu...@gmail.com> wrote:
> Thanks for your answers.
>
> As I understand Nick's response, I only need to use appengine_rpc.py
> for the entire process. I tried the following:
>
> -------
>
> def passwdFunc():
>     return ('my_email','my_passwd')
>
> rpcServer = appengine_rpc.HttpRpcServer
> ('myapp.appspot.com',passwdFunc,None,'myAppName')
> blah = rpcServer.Send('/')
>
> -------
>
> This gave me a 302 error and the following log:
>
> -------
>
> Server: myapp.appspot.com
> Sending HTTP request:
> POST /? HTTP/1.1
> Host: myapp.appspot.com
> X-appcfg-api-version: 1
> Content-type: application/octet-stream
>
> Got http error, this is try #1

> Got 302 redirect. Location:https://www.google.com/accounts/ServiceLogin?service=ah&continue=http...

[Tony Rowles]

The response has the "Set-cookie" header set, which will cause the
user's browser to save the cookie and then present it on the next
request (after redirected by the 302). In my code I've opted not to
follow the redirect, and extracted the cookie myself, because it's the
urlfetch service doing the request, not the user. You can then either
return a response to the user with a "Set-cookie" HTTP header (causing
their browser to save the cookie), or handle it some other way (return
it in the body and set the cookie with Javascript, for example).

I think I misunderstood your original question, though, and you're
looking for something different. You want to get an authorization
cookie and then use it to make repeated requests with urlfetch, not
with a browser? If that's the case, you're going to want to capture
the "Set-cookie" header from the second response, and supply that in
future requests (setting the "Cookie" header for urlfetch).

Basically, urlfetch will follow redirects but it won't handle cookies
automatically - so what's happening is it's ignoring the "Set-cookie"
header and following the redirect, and being denied because it's not
supplying a cookie.

On Jul 15, 2:58 pm, epb <esbenbu...@gmail.com> wrote:
> I can see why Tony's version would work. His "algorithm" has two
> steps:
>
> 1. Get the authorization token using ClientLogin (which I also managed
> to do).
> 2. Use the uri "servername/_ah/login" to get the auth. cookie.
>
> The appengine_rpc module seems to do authentication in a similar way:
>
> A. Try to access the app. This results in a redirect to a location

> that starts withhttps://www.google.com/accounts/ServiceLogin

[epb]

On Jul 15, 4:05 pm, Tony <fatd...@gmail.com> wrote:
> The response has the "Set-cookie" header set, which will cause the
> user's browser to save the cookie and then present it on the next
> request (after redirected by the 302).  In my code I've opted not to
> follow the redirect, and extracted the cookie myself, because it's the
> urlfetch service doing the request, not the user.  You can then either
> return a response to the user with a "Set-cookie" HTTP header (causing
> their browser to save the cookie), or handle it some other way (return
> it in the body and set the cookie with Javascript, for example).
>
> I think I misunderstood your original question, though, and you're
> looking for something different.  You want to get an authorization
> cookie and then use it to make repeated requests with urlfetch, not
> with a browser?  If that's the case, you're going to want to capture
> the "Set-cookie" header from the second response, and supply that in
> future requests (setting the "Cookie" header for urlfetch).

Yes, that is exactly what I want :) My client app. is not browser-
based. I guess I'll just use your method then.. appengine_rpc must be
intended for browser apps only, as it does nothing to capture the
auth. cookie. I could of course extend the appengine_rpc module to
capture the cookie, but the module uses urllib2.OpenerDirector.open()
to open URLs and this is perhaps not the way to go in my case? I am
not sure what the difference is between urlfetch() and open().... it
seems like I can get the headers (and hereby the cookie) by using info
() on the response from open().

[Tony Rowles]

I believe that the difference between urlfetch and urllib2 is
superficial - App Engine makes all requests using urlfetch, regardless
of which lib you use in your code.

[Nick Johnson (Google)]

appengine_rpc is intended for any Python app. It captures the cookie
by using a CookieJar, which does the capturing/sending of the cookie.

-Nick Johnson

[epb]

Nick, I tried running the following code, having added
save_cookies=True

-----

def passwdFunc():
return ('my_email','my_passwd')

rpcServer = appengine_rpc.HttpRpcServer

('myapp.appspot.com',passwdFunc,None,'myAppName',save_cookies=True)

blah = rpcServer.Send('/')

-----

It still gives me a 302 error:

-----

Server: myapp.appspot.com
Sending HTTP request:
POST /? HTTP/1.1
Host: myapp.appspot.com
X-appcfg-api-version: 1
Content-type: application/octet-stream


Got http error, this is try #1
Got 302 redirect. Location:

https://www.google.com/accounts/ServiceLogin?service=ah&continue=http://myapp.appspot.com/_ah/login%3Fcontinue%3Dhttp://myapp.appspot.com/&ltmpl=gm&ahname=myapp&sig=ed2431f623b91........8cb98a1014
Saving authentication cookies to ~/.appcfg_cookies

Sending HTTP request:
POST /? HTTP/1.1
Host: myapp.appspot.com
X-appcfg-api-version: 1
Content-type: application/octet-stream


Got http error, this is try #2

-----

Trying it again it now loads the cookies like it's supposed to, but I
still get a redirect in the end (now to myapp.appspot.com??):

-----

Loaded authentication cookies from ~/.appcfg_cookies

Server: myapp.appspot.com
Sending HTTP request:
POST /? HTTP/1.1
Host: myapp.appspot.com
X-appcfg-api-version: 1
Content-type: application/octet-stream


Got http error, this is try #1

Got 302 redirect. Location: http://myapp.appspot.com/

Sending HTTP request:
POST /? HTTP/1.1
Host: myapp.appspot.com
X-appcfg-api-version: 1
Content-type: application/octet-stream


Got http error, this is try #2

-----

Please, let me know what I am doing wrong. Thanks in advance!

On Jul 16, 9:23 am, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:

[epb]

Anyone who can help me on this? Like mentioned above I am able to save
the cookie using the appengine_rpc module, but the saved cookie is not
used properly as I am redirected again :(

I tried the custom version too, using Tony's code to get the cookie
and capture it in memory. But when I try to access the server with the
cookie in my CookieJar, the login-page is returned. My code:

cookiejar = cookielib.CookieJar()
request = urllib2.Request(target)
cookiejar.extract_cookies(response, request)
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookiejar))
f = opener.open(request)

As mentioned, a handle to the login-page is returned in the variable
'f'. If I print the cookiejar I can see the cookie, so it seems that
the cookie is fetched alright:
>>>print cookiejar
<cookielib.CookieJar[<Cookie ABCD=abcdefghijkl12345 for
myapp.appspot.com/accounts>]>

Do I need to send the cookie to the server in another way? Sorry about
all these questions, but I can't seem to find documentation on this..

On Jul 16, 11:22 am, epb <esbenbu...@gmail.com> wrote:
> Nick, I tried running the following code, having added
> save_cookies=True
>
> -----
>
> def passwdFunc():
>     return ('my_email','my_passwd')
>
> rpcServer = appengine_rpc.HttpRpcServer
> ('myapp.appspot.com',passwdFunc,None,'myAppName',save_cookies=True)
> blah = rpcServer.Send('/')
>
> -----
>
> It still gives me a 302 error:
>
> -----
>
> Server: myapp.appspot.com
> Sending HTTP request:
> POST /? HTTP/1.1
> Host: myapp.appspot.com
> X-appcfg-api-version: 1
> Content-type: application/octet-stream
>
> Got http error, this is try #1

> Got 302 redirect. Location:https://www.google.com/accounts/ServiceLogin?service=ah&continue=http...