Task Queue API Users

[hawkett]

Hi,

I'm running into some issues trying to use the Task Queue API with
restricted access URL's defined in app.yaml - when a URL is defined as
either 'login: admin' or 'login: required', when the task fires it is
receiving a 302 - which I assume is a redirect to the login page. I'm
just running this on the SDK at the moment, but I was expecting at
least the 'login: admin' url to work, based on the following comment
from this page http://code.google.com/appengine/docs/python/taskqueue/overview.html

'If a task performs sensitive operations (such as modifying important
data), the developer may wish to protect the worker URL to prevent a
malicious external user from calling it directly. This is possible by
marking the worker URL as admin-only in the app configuration.'

I figure I'm probably doing something dumb, but I had expected the
tasks to be executed as some sort of system user, so that either
'login: required' or 'login: admin' would work - perhaps even being
able to specify the email and nickname of the system user as app.yaml
configuration. Another alternative would be if there was a mechanism
to create an auth token to supply when the task is created. e.g.
users.current_user_auth_token() to execute the task as the current
user.

So I guess the broader question is - where does the task queue get the
'run_as' user, or if there isn't one, what's the mechanism for hitting
a 'login: admin' worker URL?

Most apps should be able to expect a call to users.get_current_user()
to return a user object in code protected by 'login: admin'.

Thanks,

Colin

[Nick Johnson (Google)]

Hi hawkett,

In the current release of the SDK, the Task Queue stub simply logs tasks to be executed, and doesn't actually execute them. How are you executing these tasks?

-Nick Johnson

--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047

[hawkett]

Hi Nick,

In my SDK (just the normal mac download), I can inspect the queue in
admin console, and have a 'run' and 'delete' button next to each task
in the queue. When I press 'run', the task fires, my server receives
the request, and returns the 302.

Colin

On Jun 22, 4:15 pm, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:

[Nick Johnson (Google)]

Hi hawkett,

My mistake. This sounds like a bug in the SDK - can you please file a bug?

-Nick Johnson

[hawkett]

Sure - just before I do, the following may indicate that this isn't a
bug -

If I am also logged in to the application in another tab, as an
administrator, then when I hit 'run' the task fires successfully, or
at least the stub fires and records a 200. So it would appear I need
to also be logged in as an admin. While this makes some sense, it
doesn't really mirror the behaviour on GAE, as the task queue won't
have the benefit of this authentication cookie - what user does the
live system use to execute protected URL's?

Colin

On Jun 22, 4:31 pm, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:
> Hi hawkett,
>

[hawkett]

Hi,

I've deployed an app to do some tests on live app engine, and the
following code

currentUser = users.get_current_user()
if currentUser is not None:
logging.info("Current User - ID: %s, email: %s, nickname: %s" %
(currentUser.user_id(), currentUser.email(), currentUser.nickname()))

logging.info("is admin? %s" % users.is_current_user_admin())

yields: 'is admin? False'

as the total log output. This is code that is run directly from a
handler in app.yaml that specified - 'login:admin'

This represents a pretty big problem - it means you can't rely on
'login:admin' to produce a user that is an admin. I'm guessing that
the goal of the Task Queue API is to be usable on generic URLs - e.g.
in a RESTful application, the full CRUD (and more) functionality is
exposed via a dynamic set of URL's that more than likely are not
specifically for the Task Queue API - however the above situation
means you really have to code explicitly for the Task Queue API,
because the meaning of the directives in app.yaml is not reliable. It
looks like cron functionality works like this as well, and that has
been around for a while. Use cases such as write-behind outlined in
Brett's IO talk are significantly limited by being unable to predict
whether you will get a user or not (especially if you intend to hit
RESTful URI that could just as easily be hit by real users). Sure,
there are ways to code around it, but it's not pretty.

I've added a defect to the issue tracker here -
http://code.google.com/p/googleappengine/issues/detail?id=1742

I'm keen to understand how google sees this situation, and whether the
current situation is here to stay, or something short term to deliver
the functionality early. Cheers,

Colin

On Jun 22, 4:31 pm, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:
> Hi hawkett,
>

[Nick Johnson (Google)]

Hi hawkett,

The bug you found earlier, with Task Queue accesses returning 302s instead of executing correctly, is definitely a bug in the dev_appserver. Can you please file a bug on the issue tracker?

On Mon, Jun 22, 2009 at 11:18 PM, hawkett <haw...@gmail.com> wrote:

Hi,

  I've deployed an app to do some tests on live app engine, and the
following code

currentUser = users.get_current_user()
if currentUser is not None:
  logging.info("Current User - ID: %s, email: %s, nickname: %s" %
(currentUser.user_id(), currentUser.email(), currentUser.nickname()))

logging.info("is admin? %s" % users.is_current_user_admin())

yields:  'is admin? False'

as the total log output.  This is code that is run directly from a
handler in app.yaml that specified - 'login:admin'

This represents a pretty big problem - it means you can't rely on
'login:admin' to produce a user that is an admin.  

On the contrary - only administrators and the system itself (eg, cron and task queue services) will be able to access "login: admin" handlers. However, when access is by a service, no user is specified, so "is_current_user_admin()" will naturally return False, not because it's not an admin access, but because there's no current user.
 

I'm guessing that
the goal of the Task Queue API is to be usable on generic URLs - e.g.
in a RESTful application, the full CRUD (and more) functionality is
exposed via a dynamic set of URL's that more than likely are not
specifically for the Task Queue API - however the above situation
means you really have to code explicitly for the Task Queue API,
because the meaning of the directives in app.yaml is not reliable.  It
looks like cron functionality works like this as well, and that has
been around for a while.  Use cases such as write-behind outlined in
Brett's IO talk are significantly limited by being unable to predict
whether you will get a user or not (especially if you intend to hit
RESTful URI that could just as easily be hit by real users).  Sure,
there are ways to code around it, but it's not pretty.

I'm not sure I see the problem - what user would you expect to see listed when a webhook is being called by the cron or task queue system?

-Nick Johnson

[hawkett]

Hi Nick,

Bug filed - http://code.google.com/p/googleappengine/issues/detail?id=1751

> I'm not sure I see the problem - what user would you expect to see listed
> when a webhook is being called by the cron or task queue system?

The problem is that the handler code needs to have an understanding of
the particular calling client. This tightly couples the handler code
to the calling mechanism. I totally wrecks the idea that the protocol
should allow loose coupling of the two end points. From my
perspective, that's bad architecture. If I explicitly say I need a
user (admin or otherwise) to access a URI, then the system should make
sure that URI is not accessed unless there is a user. Once you start
introducing edge cases - 'It's true unless this, or unless that', the
platform becomes 'clunky'. app.yml is an interface contract, and
currently asynch breaks that contract. That contract is far more
important than one client's (GAE system) difficulty (which user?)
conforming to it. My 2c anyway. Thanks,

Colin

On Jun 23, 10:46 am, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:
> Hi hawkett,
>

[Tony Rowles]

I think (correct me if I'm wrong) that what Colin is saying is that if
User A is logged in, and performs an action on a page which enqueues a
task, and the task hits a webhook, the webhook should be able to
operate just as if User A had logged in, and hit the webhook url (so
users.get_current_user() should return the user that enqueued the
task).

The workaround seems pretty easy, though, just pass the required
information in the payload: "if user is None: user = db.get(request.get
('userkey'))," or "if user is None: username = db.get(request.get
('username'))" or what have you.

Or maybe he's just saying you should be able to assign more granular
permissions like:

- url: /hook
login: [admin, cron]

Or maybe I'm missing his point entirely :P

On Jun 23, 9:02 am, hawkett <hawk...@gmail.com> wrote:
> Hi Nick,
>

>   Bug filed -http://code.google.com/p/googleappengine/issues/detail?id=1751

>
> > I'm not sure I see the problem - what user would you expect to see listed
> > when a webhook is being called by the cron ortaskqueuesystem?
>
> The problem is that the handler code needs to have an understanding of
> the particular calling client.  This tightly couples the handler code
> to the calling mechanism.  I totally wrecks the idea that the protocol
> should allow loose coupling of the two end points.  From my
> perspective, that's bad architecture.  If I explicitly say I need a
> user (admin or otherwise) to access a URI, then the system should make
> sure that URI is not accessed unless there is a user.  Once you start
> introducing edge cases - 'It's true unless this, or unless that', the
> platform becomes 'clunky'. app.yml is an interface contract, and
> currently asynch breaks that contract. That contract is far more
> important than one client's (GAE system) difficulty (which user?)
> conforming to it.  My 2c anyway.  Thanks,
>
> Colin
>
> On Jun 23, 10:46 am, "Nick Johnson (Google)" <nick.john...@google.com>
> wrote:
>
>
>
> > Hi hawkett,
>

> > The bug you found earlier, withTaskQueueaccesses returning 302s instead

> > of executing correctly, is definitely a bug in the dev_appserver. Can you
> > please file a bug on the issue tracker?
>
> > On Mon, Jun 22, 2009 at 11:18 PM, hawkett <hawk...@gmail.com> wrote:
>
> > > Hi,
>
> > >   I've deployed an app to do some tests on live app engine, and the
> > > following code
>
> > > currentUser = users.get_current_user()
> > > if currentUser is not None:
> > >   logging.info("Current User - ID: %s, email: %s, nickname: %s" %
> > > (currentUser.user_id(), currentUser.email(), currentUser.nickname()))
>
> > > logging.info("is admin? %s" % users.is_current_user_admin())
>
> > > yields:  'is admin? False'
>
> > > as the total log output.  This is code that is run directly from a
> > > handler in app.yaml that specified - 'login:admin'
>
> > > This represents a pretty big problem - it means you can't rely on
> > > 'login:admin' to produce a user that is an admin.
>
> > On the contrary - only administrators and the system itself (eg, cron and

> >taskqueueservices) will be able to access "login: admin" handlers.

> > However, when access is by a service, no user is specified, so
> > "is_current_user_admin()" will naturally return False, not because it's not
> > an admin access, but because there's no current user.
>
> > > I'm guessing that

> > > the goal of theTaskQueueAPI is to be usable on generic URLs - e.g.

> > > in a RESTful application, the full CRUD (and more) functionality is
> > > exposed via a dynamic set of URL's that more than likely are not

> > > specifically for theTaskQueueAPI - however the above situation

> > > > > in thequeue.  When I press 'run', thetaskfires, my server receives

> > > > > the request, and returns the 302.
>
> > > > > Colin
>
> > > > > On Jun 22, 4:15 pm, "Nick Johnson (Google)" <nick.john...@google.com>
> > > > > wrote:
> > > > > > Hi hawkett,
>

> > > > > > In the current release of the SDK, theTaskQueuestub simply logs

> > > tasks
> > > > > to
> > > > > > be executed, and doesn't actually execute them. How are you executing
> > > > > these
> > > > > > tasks?
>
> > > > > > -Nick Johnson
>
> > > > > > On Mon, Jun 22, 2009 at 3:46 PM, hawkett <hawk...@gmail.com> wrote:
>
> > > > > > > Hi,
>
> > > > > > >   I'm running into some issues trying to use theTaskQueueAPI
> > > with
> > > > > > > restricted access URL's defined in app.yaml - when a URL is defined
> > > as

> > > > > > > either 'login: admin' or 'login: required', when thetaskfires it

> > > is
> > > > > > > receiving a 302 - which I assume is a redirect to the login page.
> > >  I'm
> > > > > > > just running this on the SDK at the moment, but I was expecting at
> > > > > > > least the 'login: admin' url to work, based on the following
> > > comment
> > > > > > > from this page
>
> > >http://code.google.com/appengine/docs/python/taskqueue/overview.html
>

> > > > > > > 'If ataskperforms sensitive operations (such as modifying

> > > important
> > > > > > > data), the developer may wish to protect the worker URL to prevent
> > > a
> > > > > > > malicious external user from calling it directly. This is possible
> > > by
> > > > > > > marking the worker URL as admin-only in the app configuration.'
>
> > > > > > > I figure I'm probably doing something dumb, but I had expected the
> > > > > > > tasks to be executed as some sort of system user, so that either
> > > > > > > 'login: required' or 'login: admin' would work - perhaps even being
> > > > > > > able to specify the email and nickname of the system user as
> > > app.yaml
> > > > > > > configuration.  Another alternative would be if there was a
> > > mechanism

> > > > > > > to create an auth token to supply when thetaskis created.  e.g.
> > > > > > > users.current_user_auth_token() to execute thetaskas the current

[hawkett]

Hi Tony,

That's the second of the options I listed in the defect/feature
request

http://code.google.com/p/googleappengine/issues/detail?id=1742

The first is to have the system run as a 'special' user specified in
app.yaml.

Another way of looking at the problem, perhaps, is that the current
asynch implementation is asking us to change this code

if users.is_current_user_admin():
# Do privileged stuff

to this

currentUser = users.get_current_user()
if users.is_current_user_admin() or currentUser is None:
# Do privileged stuff

i.e. no user = admin user, and that smells funny to me. I'm also
thinking we need to be able to call URL's protected in app.yaml with
'login: required', as this is going to be a common use case - a place
where many app's urls naturally reside. Thanks,

Colin

[bFlood]

i agree with colin on the "no user = admin" fishy-ness. it would be
nice if the task system just set the current user to admin. that way,
if for some reason your task url endpoint got out on the web (although
not likely), you could safely ignore the requests with no user. right
now you'd have no way of telling (hmm, request headers or ip address
might do)

for running tasks under the user who started them, I would agree that
adding this info to the payload is sufficient

cheers
brian

[Nick Johnson (Google)]

Hi bFlood,

On Thu, Jun 25, 2009 at 4:18 PM, bFlood<bflo...@gmail.com> wrote:
>
> i agree with colin on the "no user = admin" fishy-ness. it would be
> nice if the task system just set the current user to admin.

Which one? App Engine apps can have multiple administrators.

> that way,
> if for some reason your task url endpoint got out on the web (although
> not likely), you could safely ignore the requests with no user. right
> now you'd have no way of telling (hmm, request headers or ip address
> might do)

You can simply set your task queue endpoint as "login: admin" in your
app.yaml. This will prevent external users from accessing it at all.
The ability of the Task Queue and Cron APIs to access "login: admin"
URLs without an admin user's credentials is simply a way to allow
users to easily secure their endpoints.

-Nick Johnson

[bFlood]

ahhh, right you are Nick. ok, so I guess its safe to say that only one
calling your task url is either a valid admin or the task framework
itself. cool

cheers
brian

On Jun 25, 11:21 am, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:
> Hi bFlood,

> >> > > > > > > > > able to specify the email and nickname...
>
> read more »