Implementation of Object Permissions

[Moritz S.]

Hi all,

I have been using django for a while and became happy by the time I discovered the auth module that supports permission handling. But this module turned out to only be able to handle model based permissions. I could not imagine django of not having such a basic functionality (in my opinion) despite of having so much other great abilities. So I started to search for it but eventually figured out that the auth model's code was prepared for implementing this but it haven't been done yet.

So I decided to write a patch for this on my own.

Currently I'm writing this patch but there are several design decisions to make so I wanted to share this with you and hear your opinions.

First of all: I think the main problem of this implementation is to link the objects with the corresponding permissions. With model based permissions that's easy because you only have to store a permission once with the models ContentType and relate with a ManyToManyField to them and that's it. The problem with object based permissions is that every object needs to have his own set of permissions and you can't just use ForeignKeys or ManyToManyFields from a permission model because in the process of creating the permission model (and table) you don't know about the models that will provide object permissions therefore you can't refer to them.

So I tried some different implementations.

At first I tried to used dynamic models (e.g. this project makes use of them: http://pypi.python.org/pypi/django-object-permissions): The idea is to create intermediate models like 'MyModel_ObjectPermissions' that stores the object permissions with foreign keys to the model itself, user and group.

But this has a huge disadvantage: the use of dynamic models.

In fact thinking as database specialist that is the easiest way of connecting the model instances to the permissions but dynamic models are very sensitive and kind of hacky in my opinion and I think that does not match well with django's ideas.

Then I found out about GenericRelations from the contenttypes framework. So you could possibly use a single model called 'ObjectPermissions' or something and link users, permissions and instances of all types of models and only had to use one GenericRelation. This solution eliminates the problem wit the dynamic models because it doesn't even make use of them. But the GenericRelations have another not negligible disadvantage: The databases behind django don't have native support for them. I thought of huge django projects handling thousands and thousands of object permissions. I think these GenericRelations wouldn't be such scalable to make use of them.

The only solution for me was to find a way to link the (unknown) model instances to the permissions in the reversed direction. So not use ForeignKeys, etc. in the permission model to refer to the unknown model but to refer from the model to the permission itself. That would fix the problem of not knowing the models while creating the permission model. So in the end I thought of following:

- there is a new model in the auth module called 'ObjectPermission' this model has ManyToManyFields to User and Group and a ForeignKey to Permission

- instead of ObjectPermission referring to the model using object permissions, a field called 'object_permissions' is added to the corresponding models (by use of the class_prepared signal) (dynamically adding fields is way better than use completely dynamic models in my opinion)

- the object_permissions field is a ManyToManyField, that refers to ObjectPermission. In fact, a 'ManyToOneField' would be sufficient but django only has one to many fields (the ForeignKeyField) and we can't just use this in the reversed direction, as I mentioned above

This picture illustrates the relations:

Furthermore I thought of slightly modifying the auth module (apart from the ObjectPermission model) in the following way:

- when the auth app is initialized a function is registered to listen to the class_prepared signal. This function adds the object_permissions field to each model, that has the Meta.object_permissions set to True (for this to work, this meta attribute has to be set in django.db.models.options)

- the User model gets the methods grant_perm and revoke_perm that takes the permission and as optional argument the object in order to simplify the process of handling permissions

- in the ModelBackend the methods has_perm, etc. have to be modified to be able to handle function calls with given objects (obviously)

So that was the technical part.

Now I want you to give feedback if this implementation is viable and maybe sometimes may get into django.

Thanks,

Moritz

[Mike Axiak]

How does it compare to the solutions that already exist listed here?

 http://djangopackages.com/grids/g/perms/

-Mike

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/WbQ6EMVuxqkJ.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Moritz S.]

I checked the four most used solutions.

The first three ones use GenericKeys (in my opinion not the ideal solution as I described above) and the fourth one as far as I can see does not use the auth permission backend but does permission handling on his own.

-Moritz

-Mike

To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to django-developers+unsubscribe@googlegroups.com.

[Aymeric Augustin]

Hi Moritz,

Thanks for your proposal! Here's my feedback.

On 14 avr. 2012, at 01:56, Moritz S. wrote:
> I found out about GenericRelations from the contenttypes framework. So you could possibly use a single model called 'ObjectPermissions' or something and link users, permissions and instances of all types of models and only had to use one GenericRelation. This solution eliminates the problem wit the dynamic models because it doesn't even make use of them.

This solution is the "simplest one that could possibly work", and as such, it's my favorite.

> But the GenericRelations have another not negligible disadvantage: The databases behind django don't have native support for them. I thought of huge django projects handling thousands and thousands of object permissions. I think these GenericRelations wouldn't be such scalable to make use of them.

With an appropriate index, SQL databases should be able to find rows matching a condition on two columns about as quickly as if the condition was only on one column, even in a table containing million of objects.

I'd like to see a benchmark before rejecting this idea on performance grounds. Could you point us to some data supporting your assertion that this isn't scalable?

> The only solution for me was to find a way to link the (unknown) model instances to the permissions in the reversed direction. So not use ForeignKeys, etc. in the permission model to refer to the unknown model but to refer from the model to the permission itself. That would fix the problem of not knowing the models while creating the permission model. So in the end I thought of following:
> - there is a new model in the auth module called 'ObjectPermission' this model has ManyToManyFields to User and Group and a ForeignKey to Permission
> - instead of ObjectPermission referring to the model using object permissions, a field called 'object_permissions' is added to the corresponding models (by use of the class_prepared signal) (dynamically adding fields is way better than use completely dynamic models in my opinion)
> - the object_permissions field is a ManyToManyField, that refers to ObjectPermission. In fact, a 'ManyToOneField' would be sufficient but django only has one to many fields (the ForeignKeyField) and we can't just use this in the reversed direction, as I mentioned above

I'm -1 on a contrib app (django.contrib.auth) injecting a dynamic field into every model.

(Yes, Django does that for the primary key, to avoid declaring it in every model. But it's core, not a contrib app.)

> Furthermore I thought of slightly modifying the auth module (apart from the ObjectPermission model) in the following way:
> - when the auth app is initialized a function is registered to listen to the class_prepared signal. This function adds the object_permissions field to each model, that has the Meta.object_permissions set to True (for this to work, this meta attribute has to be set in django.db.models.options)

What's the definition of "when the auth app is initialized"? Wouldn't your solution required that django.contrib.auth be loaded before all other apps?

> - the User model gets the methods grant_perm and revoke_perm that takes the permission and as optional argument the object in order to simplify the process of handling permissions

With the solution that I prefer, this doesn't seem absolutely necessary, the following statements are simple enough:

ObjectPermission.objects.create(user=user, object=instance)
ObjectPermission.objects.get(user=user, object=instance).delete()

> - in the ModelBackend the methods has_perm, etc. have to be modified to be able to handle function calls with given objects (obviously)

Yes.

Best regards,

--
Aymeric.

[Λlisue]

Did you check my library ( django-permission: https://github.com/lambdalisue/django-permission )? which is not stable yet and has completely different approach to enable Object Permission in Django.

I gave up database based object permission. If you have some article which is only visible for authenticated user
, you may want to give a "view" permission for all authenticated users when the article has saved (if you don't use
"is_authenticated" or "login_required" or whatever for some reason). That works fine while you add new user. If you
add new user then you have to give permission of the article to the new user. If you have a lot of user and a lot of article, objects...
adding new user takes long time and it's not efficient.

Just in case if my library helps you. Regard.

To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/EzRqqRU1eSUJ.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.

[Daniel Sokolowski]

Mortiz, have you taken a look at django-guardian, it has been out as long as object level hooks have been implemented; I have been using it successfully since version 0.2 which is about 2 years now.

 

Your need to add ‘object_permissions’ to a model is an incorrect approach in my opinion as it would require any old apps to undergo a schema migration. Your generic relations concerns might be valid however I would like to mention Django guardian does use generic FK but so does built in django auth – so I think this approach is just fine.

 

In the end I would humbly suggest that rather then creating a competing solution you instead contribute your skills and suggestions to an already working project.

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/WbQ6EMVuxqkJ.

To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

 

Daniel Sokolowski
Danols Web Engineering

http://webdesign.danols.com/
Kingston, ON K7L 1H3, Canada


Notice of Confidentiality:
The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review re-transmission dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you received this in error please contact the sender immediately by return electronic transmission and then immediately delete this transmission including all attachments without copying distributing or disclosing same.

[Moritz S.]

With an appropriate index, SQL databases should be able to find rows matching a condition on two columns about as quickly as if the condition was only on one column, even in a table containing million of objects.

Databases can of course find the rows with appropriate indexes, but the problem is that GenericForeignKeys are precisely not such appropriate ones. 

I'd like to see a benchmark before rejecting this idea on performance grounds. Could you point us to some data supporting your assertion that this isn't scalable?

To prove the increased performance I actually did some benchmark. I used the script in the attachment. It needs a model that has only one field called text. It randomly generates 10,000 users and 1,000,000 objects. Then it assigns to each user respectively 1,000 object permissions of randomly selected previously created objects. At least it checks for each user respectively 1,000 randomly selected objects again. That's what I got:

For the first version of my contrib.auth patch:

Summary:

granted permissions in 1 day, 10:41:07.523402

checked permissions in 22:48:44.662974

For a module using GenericForeignKeys (I used django-guardian because it seemed to be the most used one):

granted permissions in 11:31:12.216808

checked permissions <pending>

I don't have the results for checking permissions with django-guardian yet because it took ages to check them. Each single check for a object permission took about 4 to 5 seconds! Now think about how long it would take to check for all users (10,000*1,000*4secs)! I think that is out of all proportion to my suggested solution.

This happens because of the lack of a appropriate index as you mentioned before. So the database has to do a full table scan and that's definitely not scalable at all.

By the way, the reason for the GenericForeignKeys being faster in creating them is that the database does not have to create the indexes (i.e. the foreign keys). 

I'm -1 on a contrib app (django.contrib.auth) injecting a dynamic field into every model.

(Yes, Django does that for the primary key, to avoid declaring it in every model. But it's core, not a contrib app.)

I totally agree with this but I don't see another way at the moment. 

What's the definition of "when the auth app is initialized"? Wouldn't your solution required that django.contrib.auth be loaded before all other apps?

Unfortunately it does but I have a open mind about better implementations.

With the solution that I prefer, this doesn't seem absolutely necessary, the following statements are simple enough:

ObjectPermission.objects.create(user=user, object=instance)
ObjectPermission.objects.get(user=user, object=instance).delete()

I think that's matter of taste ;-) 

Your need to add ‘object_permissions’ to a model is an incorrect approach in my opinion as it would require any old apps to undergo a schema migration. Your generic relations concerns might be valid however I would like to mention Django guardian does use generic FK but so does built in django auth – so I think this approach is just fine.

You don't have to add this field manually, auth app does that. And django auth does not use generic relationships, it only makes use of the ContentType field to comfortably store the type assigned to a permission. 

Best regards,

Moritz

[Daniel Sokolowski]

Fantastic job on the performance benchmarks, the number 4-5 seconds per check is scary. I’ve posted a comment on django-guradian in hopes the author can chime in onto this conversation by posting https://groups.google.com/forum/#!topic/django-developers/6UNjPu1mcgc/discussion. 

 

Are there any hopes of speeding up or caching GenericForgeinKey queries ? I found -  http://zerokspot.com/weblog/2008/08/13/genericforeignkeys-with-less-queries/ – can your solution be backward compatible?

 

And yes you are correct there is no GenericForeginKey on the Permission model, my bad; at the moment my understanding of this limits me really to just the opinion level.

--

You received this message because you are subscribed to the Google Groups "Django developers" group.

To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/fp-Bu8sgUq0J.

[Moritz S.]

Are there any hopes of speeding up or caching GenericForgeinKey queries ? I found -  http://zerokspot.com/weblog/2008/08/13/genericforeignkeys-with-less-queries/ 

Adding 'db_index=True' to the GenericForeignKey's object_id field should be a significant performance enhancement. It won't get as good results as the other solution, though, but that should prevent the database from doing full table scans.

But I'm not sure whether GenericRelations allow this, because no django object permission app does this. 

can your solution be backward compatible?

My solution should be backward compatible as-is. As default, the patch assumes every model not having the ability to grant/check for object permissions. So it won't get in conflict with those (outdated) models. Furthermore it does not modify the existing contrib.auth models in a way that would require a new database table schema but only adds a new model called ObjectPermission.

Best regards,

Moritz

[Lukasz Balcerzak]

GFK approach is scary, indeed. I've got django-guardian to life during the weekend for some future, rather simple, business apps (aimed to be used within company's intranet). I've made one mistake that is already filed (https://github.com/lukaszb/django-guardian/issues/11). Didn't ever really started working on it, though, as I still feel it's not a long-run solution. Removing Generic Foreign Keys - or at least letting people to easy use FKs instead - is much better one.

However, it should be fairly simply to avoid forcing GFK and I was thinking about this since I first saw some talk by Alex quite long time ago - I believe he's approach at https://github.com/alex/django-taggit is much more desirable for developers (explicit, customizable, providing better performance). By incorporating similar technique django-guardian (and other libraries) could get much better results.

The problem is - I don't have much time recently to work on django-guardian (or even give some answer to this thread) so if someone has the energy to contribute, please do! I've also created proper ticket at issue tracker (https://github.com/lukaszb/django-guardian/issues/69)

To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to django-developers+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.