[Django] #23422: Cannot add Permission to Group in data migration
Django
----------------------------+--------------------
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.7
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------+--------------------
As initial_data fixtures are now deprecated I want to move my things there
to migration files instead. One thing I do is to create auth.Group(s) and
give these some custom permissions that I have defined on the app's
models. This does not seem to be possible as, from what I can tell, the
permissions and content types are created in a signal AFTER all the
Migrations are run. Neither custom nor standard permissions can be found.
A Group and its permissions are data and the initialization thereof should
therefore be possible to achieve through the migration system since its
supposed to replace the fixtures.
{{{
my_new_group.permissions.add(
Permission.objects.get_by_natural_key('add_my_model', 'my_app',
'my_model')
)
}}}
This gives:
{{{
django.contrib.contenttypes.models.DoesNotExist: ContentType matching
query does not exist.
}}}
even though I created the model in the previous migration step. There are
no ContentTypes ate all yet.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.7
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by chriscauley):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
I'm a bit new to the migrations system, but I was able to verify this. I
created a new app called "my_app" and a model called "MyModel". I created
the first migration and then the following data migration:
{{{
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
def make_permissions(apps,schema_editor):
Group = apps.get_model("auth","Group")
Permission = apps.get_model("auth","Permission")
my_new_group = Group(name="test_group")
my_new_group.save()
my_new_group.permissions.add(
Permission.objects.get(codename='add_mymodel',content_type__app_label='my_app')
)
class Migration(migrations.Migration):
dependencies = [
('my_app', '0001_initial'),
('auth', '0001_initial'),
]
operations = [
migrations.RunPython(make_permissions,reverse_code=lambda
*args,**kwargs: True)
]
}}}
If I run these separately (`python manage.py migrate my_app 0001;python
manage.py migrate my_app 0002`), everything is cool. Together it breaks
(`python manage.py migrate my_app`). You could solve this yourself by
doing a get_or_create on the content type and on the permission, but that
is by no means an adequate solution.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:1>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by andrewgodwin):
* status: new => closed
* resolution: => wontfix
Comment:
This is a limitation of the signalling system we use to create
ContentTypes; if we fixed this to work, there would then be issues where
the ContentTypes were created too early.
I would suggest that the data migration uses
ContentType.objects.get_for_model to get the ContentType before making the
Permission, as this creates the ContentType if necessary and has internal
caching too.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:2>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Nicals):
I understand the won't fix, but a way to solve this group permissions
problem would be appreciated. I'm blocked on a project by this...
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:3>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by coagulant):
I figure out ContentType.objects.get_for_model won't work "as is" in
datamigration, because only default model managers are available.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:4>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by carljm):
If using `ContentType.objects.get_for_model` is not possible, perhaps
calling `django.contrib.auth.management.create_permissions(...)` in the
migration would work?
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:5>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by jwhitlock):
Here's an example of calling `create_permissions` directly from a data
migration, as suggested by carljm:
{{{
def make_permissions(apps, schema_editor, with_create_permissions=True):
Group = apps.get_model("auth", "Group")
Permission = apps.get_model("auth", "Permission")
try:
perm = Permission.objects.get(
codename='add_mymodel', content_type__app_label='my_app')
except Permission.DoesNotExist:
if with_create_permissions:
# Manually run create_permissions
from django.contrib.auth.management import create_permissions
assert not getattr(apps, 'models_module', None)
apps.models_module = True
create_permissions(apps)
apps.models_module = None
return make_permissions(
apps, schema_editor, with_create_permissions=False)
else:
raise
my_new_group = Group.objects.create(name="test_group")
my_new_group.permissions.add(perm)
}}}
Feels a little hacky, but it solved this `wontfix` for me . I'd prefer a
migration configuration that says "stop migrating, emit `post_migrate`
signals, and restart migrations."
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:6>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by bronger):
South used to have `send_pending_create_signals()`. A django equivalent
would be useful.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:7>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by bronger):
* cc: bronger@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:8>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by collinanderson):
One thing we could do is have a `post_migration` signal that runs after
_every_ migration. That way it's consistent.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:9>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by MarkusH):
You might want to keep an eye on #24100
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:10>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by scottwoodall):
* cc: scott.woodall@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:11>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by guettli):
I am not happy with this issue being "wontfix".
A helper method `create_missing_content_types_and_permissions()` would be
nice.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:12>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by guettli):
* cc: hv@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:13>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by TZanke):
* cc: TZanke (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:14>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by dmugtasimov):
I am not happy with this issue being "wontfix", too.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:15>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by antialiasis):
Chiming in with another +1 on the not-wontfixing - this is stopping me
from running tests.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:16>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by frnhr):
* cc: frnhr (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:17>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by frnhr):
The solution jwhitlock proposes with calling `create_permissions` from
migration fixed one of my problematic migration, but not another. The
other migration is calling `loaddata` command to do a one-time fixture
import, and it was failing because required `ContentType` objects haven't
been created. So it seems to be another face of the same problem.
Here is a more dirty workaround until an official solution becomes
available. I think it has better changes of fixing the problem in case
solution with `create_permissions` fails:
{{{
if ContentType.objects.filter(**whats_needed_for_this_migration).count() <
1:
print("Running another migrate command from migration to create
ContentType objects:")
call_command('migrate', 'users', '0002') # this should cite migration
from the same app, just prior to this one
print("Finished with the migrate command.")
}}}
I'm guessing the only thing really needed from the `migrate` management
command in this situation is:
{{{
emit_post_migrate_signal(created_models, self.verbosity, self.interactive,
connection.alias)
}}}
but not sure how to populate the arguments.
I have no idea if using `call_command("migrate")` from a migration (of all
places...) can be problematic, but in my particular case there don't seem
to be any problems.
The lengths we have to go to... Please **willfix** this.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:18>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by adehnert):
* cc: adehnert (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:19>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by alanjds):
* needs_docs: 0 => 1
Comment:
Replying to [comment:12 guettli]:
> I am not happy with this issue being "wontfix".
>
> A helper method `create_missing_content_types_and_permissions()` would
be nice.
Found! This helper method exists already:
`django.core.management.sql.emit_post_migrate_signal`
{{{
from django.core.management.sql import emit_post_migrate_signal
def create_group(apps, schema_editor):
db_alias = schema_editor.connection.alias
try:
emit_post_migrate_signal(2, False, 'default', db_alias)
except TypeError: # Django < 1.8
emit_post_migrate_signal([], 2, False, 'default', db_alias)
Group = apps.get_model('auth', 'Group')
Permission = apps.get_model('auth', 'Permission')
(...)
}}}
I suggest this behavior and "how to deal with" to be documented on the
Permissions page.
Is this acceptable instead of a ''wontfix'' ?
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:20>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Changes (by alanjds):
* cc: alan.justino@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:21>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Changes (by blueyed):
* cc: django@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:22>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Comment (by cmichal):
Replying to [comment:20 alanjds]:
>
> Is this acceptable instead of a ''wontfix'' ?
way better. it works for me, thanks!
one minor thing is that when you add group permission for the first time
you need to pass it's id, for the second time Permission object is also
good - strange...
m
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:23>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Comment (by acidjunk):
I can confirm that it also works for me.
0001.py added custom permissions
0002.py assigned custom permissions to a group
Without the workaround by guettli in 0002.py: the custom permissions
weren't available.
Thanks
R
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:24>
Django
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
Comment (by theoden-dd):
Replying to [comment:20 alanjds]:
I've failed with Django 1.9, even after adapting emit_post_migrate_signal
to its new signature:
{{{
db_alias = schema_editor.connection.alias
try:
# Django 1.9
emit_post_migrate_signal(2, False, db_alias)
except TypeError:
# Django < 1.9
try:
# Django 1.8
emit_post_migrate_signal(2, False, 'default', db_alias)
except TypeError: # Django < 1.8
emit_post_migrate_signal([], 2, False, 'default', db_alias)
}}}
This led me to this most significant part of stack trace and error:
{{{
...
File "<venv>/local/lib/python2.7/site-
packages/django/core/management/sql.py", line 50, in
emit_post_migrate_signal
using=db)
File "<venv>/local/lib/python2.7/site-
packages/django/dispatch/dispatcher.py", line 192, in send
response = receiver(signal=self, sender=sender, **named)
File "<venv>/olivia/local/lib/python2.7/site-
packages/django/contrib/sites/management.py", line 20, in
create_default_site
if not Site.objects.using(using).exists():
...
django.db.utils.OperationalError: no such table: django_site
}}}
Has anyone got the success with 1.9?
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:25>
Django
----------------------------+--------------------------------------
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------+--------------------------------------
Comment (by charettes):
FWIW I started to work on this issue (which is related to #24100 to
#24067) on [https://github.com/django/django/compare/master...charettes
:migration-signals-refactor a branch].
The plan here is to make the `pre_migrate` signal dispatch it's plan
(#24100) and inject `CreatePermision` operations after `CreateContentType`
operations (that are injected after `CreateModel` operations just like
`RenameContentType` operations are meant to be injected after
`RenameModel` operations in order to solve #24067).
I'll try to get this into 1.10.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:26>
Django
----------------------------+--------------------------------------
Reporter: tjwalch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
----------------------------+--------------------------------------
* cc: info@… (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:27>
Django
Reporter: Tomas Walch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Floréal Cabanettes):
Replying to [comment:20 Alan Justino da Silva]:
It still works with django 2.2 with some few changes:
{{{
from django.core.management.sql import emit_post_migrate_signal
def create_group(apps, schema_editor):
emit_post_migrate_signal(2, False, 'default')
Group = apps.get_model('auth', 'Group')
Permission = apps.get_model('auth', 'Permission')
(...)
}}}
Thank you, because else, I can't use migration for that, and it should be
a problem!
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:28>
Django
-----------------------------+--------------------------------------
Reporter: Tomas Walch | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------+--------------------------------------
Comment (by wkleinheerenbrink):
The `emit_post_migrate_signal` is not a complete safe way, for this might
cause other post migrate in apps to be triggered as well that now expect
their tables to exist while they don't yet. For instance the Sites
framework will run `create_default_site` in post migrate, causing this
error.
Other solutions are proposed in:
* https://stackoverflow.com/questions/31735042/adding-django-admin-
permissions-in-a-migration-permission-matching-query-does-n
* https://stackoverflow.com/questions/57958722/django-python-loaddata-
fails-with-django-db-utils-integrityerror
Do note that these solutions required a dependency of migration
`("contenttypes", "0002_remove_content_type_name")` since Django 3.2.5.
--
Ticket URL: <https://code.djangoproject.com/ticket/23422#comment:29>