Authorization decorators always generate db queries

[Ids]

Hi,

I think we found a serious bug (at least, we hope it is not a designed
feature). If you create a simple controller (like test.py) with the
following actions:
def test():
return dict()

@auth.requires_permission('foo bar')
def test2():
return dict()

And a simple view (like test.html) with:
{{extend 'layout.html'}}
<h1>Test</h1>
{{=response.toolbar()}}

And then open the first action ( <app>/test/test ) in your browser and
inspect the db stats from the response toolbar, you'll see:
SELECT auth_permission.group_id FROM auth_permission WHERE
(((auth_permission.name = 'foo bar') AND (auth_permission.table_name =
'')) AND (auth_permission.record_id = 0));

If you comment out the decorator, there are no db queries. So this
means that somehow all decorators are executed and because the do
check the database you will get unnecessary database queries. If you
have a controller file with for example 10 actions and all of them
have a auth decorator, you will at least get 10 queries for this
simple single request. We noticed this because I had multiple
decorated actions in one controller file and we got about 30 db
queries for a simple request (like test above). Most of the queries
were of course even duplicates, because of the same decorator.

We noticed this behavior on Version 1.99.2 (2011-09-26 06:55:33)
stable and Version 1.99.3 (2011-11-16 22:36:13) dev

I think this should be fixed, because this will result in unnecessary
bad performance in larger apps.

Regards,
Ids

[Khalil KHAMLICHI]

Hi,
I think this is due to the fact of using RBAC (role based access control), which is for very secure applications, large apps are normaly not that much secured, but there are other solutions like fo example loading those tables into memory.
Khalil

[Ids]

Sure, but if I hit request A in a browser, I do not expect any
decorators on action B to hit the database. Actually, you wouldn't
expect a decorator on action B to be executed at all.... This would
potentially give lots of nasty side effects..

On Nov 18, 3:35 pm, Khalil KHAMLICHI <khamlichi.kha...@gmail.com>
wrote:

[Khalil KHAMLICHI]

I am not an expert in this, but I believe that :

@auth.requires_login()

there would be no database quieries because auth is stored in session which I believelives in

[Khalil KHAMLICHI]

Sorry pressed send button by mistake.
I was saying that I believe auth lives in memory so no db access is supposed to happen,
and I go back and say that RBAC will need db access all times.
Khalil

[Ids]

But I only expect it to hit the database i.e. execute the permission
check, when I request a "protected" action in a controller. Not if I
request an "unprotected" action.
So, using the example above, if I call/request test I DO NOT expect
database avtivity, if I call test2 I DO expect database activity.

It probably has to do with the way python decorators with parameters
work, but these side effects are really bad.

On Nov 18, 4:40 pm, Khalil KHAMLICHI <khamlichi.kha...@gmail.com>
wrote:

> Sorry pressed send button by mistake.
> I was saying that I believe auth lives in memory so no db access is
> supposed to happen,
> and I go back and say that RBAC will need db access all times.
> Khalil
>

> On Nov 18, 2011 3:37 PM, "Khalil KHAMLICHI" <khamlichi.kha...@gmail.com>

[Ids]

Maybe a better example is this:
def index():
# create public index page stuff
return dict(...)

@auth.requires_permission('administrate users')
def admin_users():
# private stuff
return dict(...)

Opening the index page would still hit the database and request
details about the 'administrate users' permission.The index page is
shown, but you still have an totally unnecessary database query. Add
more similarly protected actions in this controller file and the
request for the index page executes even more database queries.

[Ids]

Further tests reveal that decorators @auth.requires_membership and
@auth.requires_permission do hit the database and @auth.requires_login
does not hit the database. @auth.requires_login probably only checks a
session cookie.

[Khalil KHAMLICHI]

That is exactly my point, il you go beyond authenticating users, you need db access to do it.
usualy you dont need RBAC for all your actions. but if you really do, you can move auth tables to RAM (mysql)

Khalil

[howesc]

the distinction being made here is that decorators appear to be executed for methods that are not called during this request.  whether or not a query is run is not really the point, the point is that it seems that code is being executed that the user never intended.

that said, i don't know enough about decorators to know how they work under the covers.  i would tend to agree with Ids that this behavior is not what i expect, and with the auth decorators that run queries can significantly slow down an application.

can anyone comment on whether Ids and my expectation of how decorators work are correct?

thanks,

cfh

[Anthony]

The problem isn't with the decorator per se, but with the condition being evaluated. If you have:

@auth.requires_permission('read')

def secrets():

    etc.

that is equivalent to:

@auth.requires(condition=auth.has_permission('read'))

def secrets():

    etc.

In this case, the decorator takes an argument (the condition to be evaluated), and the argument itself is a function call that does a db query. So, setting up the decorated function ends up running the db query in auth.has_permission. In trunk, there is a new feature that allows a way around this -- the condition argument to auth.requires can be a callable, so you can do:

@auth.requires(lambda: auth.has_permission('read'))

def secrets():

In that case, when the decorator is created, it simply passes the lambda as the condition, rather than calling auth.has_permission. The lambda doesn't actually get called until the secrets() function is called.

Perhaps we should change auth.requires_membership and auth.requires_permission to use lambda in this way.

Anthony

[Ids]

I created issue 526 for this ( http://code.google.com/p/web2py/issues/detail?id=526#c0
). I'm not sure how the decorators are implemented, but I think
Anthony's solution using lambda points in the right direction to fix
this.

[Anthony]

I've already submitted a patch.

Anthony

[Anthony]

This has been fixed in trunk. Please try it out if you can and report back.

Anthony

On Friday, November 18, 2011 5:00:10 PM UTC-5, Ids wrote:

[Ids]

Ok. Will try it monday when I get back to work and have access to a computer with web2py.

[Ids]

Checked it just now with Version 1.99.3 (2011-11-20 21:10:53) dev
Everything now works ok, no more unnecessary database queries from
auth decorators. Thanks

[Massimo Di Pierro]

Thanks to Anthony, this is an important fix that will speed up a lot
complex apps.