[Django] #27044: `apps` passed to post_migrate_signal should have access to migrated apps even when no migration has been applied to them
Django
even when no migration has been applied to them
----------------------------+--------------------
Reporter: tkhyn | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.10
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------+--------------------
Hi,
I noticed that when one runs `migrate` without arguments (implicitly
migrating all apps), the `apps` instance passed to
`emit_post_migrate_signal`
(https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L221
[here]) contains the migrated apps' configurations only in the first run.
On subsequent runs, the `plan` list
([https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L136
here]) is empty and `post_migrate_apps` is set to `pre_migrate_apps`
([https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L197
here]), i.e. the unmigrated apps.
For example, when calling `migrate` with the `contenttypes` app enabled,
this means that `update_contenttypes`
([https://github.com/django/django/blob/1.10/django/contrib/contenttypes/management.py#L89
here]) is passed an `apps` instance with `contenttypes` missing. The
consequence is that `update_contenttypes` can not go further than
[https://github.com/django/django/blob/1.10/django/contrib/contenttypes/management.py#L102
this line].
To temporarily fix the issue and correctly trigger the
`post_migrate_signal` so that the content types are properly updated, I
need to run `flush` just after `migrate`. I don't think it should not be
needed to run `flush` simply to correctly trigger the
`post_migrate_signal` that will update the content types.
It worked fine in Django 1.9.9 as no project state apps instance is passed
to `emit_post_migrate_signal`, and `update_contenttypes` uses the global
`apps` instance.
I don't really know enough about the recent developments regarding apps
management in Django, so I'm not sure what should be done to solve this
issue. Thanks in advance if somebody has the time to look at it.
''Context: I'm testing a reusable django app ([https://bitbucket.org/tkhyn
/django-gm2m django-gm2m]) which test suite includes migration tests for a
number of mini test apps with different models (see
[https://bitbucket.org/tkhyn/django-
gm2m/src/63af327624a45c5aed110546d3d705a6b1bf7c2b/tests/?at=release
here]). Whenever I switch to another mini-app, the database needs not only
to be flushed but also re-migrated and the contenttypes need to be updated
!.''
--
Ticket URL: <https://code.djangoproject.com/ticket/27044>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.10
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by tkhyn):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Old description:
> ([https://bitbucket.org/tkhyn/django-gm2m django-gm2m]) which test suite
> includes migration tests for a number of mini test apps with different
> models (see [https://bitbucket.org/tkhyn/django-
> gm2m/src/63af327624a45c5aed110546d3d705a6b1bf7c2b/tests/?at=release
> here]). Whenever I switch to another mini-app, the database needs not
> only to be flushed but also re-migrated and the contenttypes need to be
> updated !.''
New description:
Hi,
I noticed that when one runs `migrate` without arguments (implicitly
migrating all apps), the `apps` instance passed to
`emit_post_migrate_signal`
(https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L221
[here]) contains the migrated apps' configurations only in the first run.
On subsequent runs, the `plan` list
([https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L136
here]) is empty and `post_migrate_apps` is set to `pre_migrate_apps`
([https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L197
here]), i.e. the unmigrated apps.
For example, when calling `migrate` with the `contenttypes` app enabled,
this means that after the first run `update_contenttypes`
([https://github.com/django/django/blob/1.10/django/contrib/contenttypes/management.py#L89
here]) is passed an `apps` instance with `contenttypes` missing. The
consequence is that `update_contenttypes` can not go further than
[https://github.com/django/django/blob/1.10/django/contrib/contenttypes/management.py#L102
this line].
To temporarily fix the issue and correctly trigger the
`post_migrate_signal` so that the content types are properly updated each
time `migrate` is called (and not only the first time), I need to run
`flush` just after `migrate`. I don't think it should not be needed to run
`flush` simply to correctly trigger the `post_migrate_signal` that will
update the content types.
It worked fine in Django 1.9.9 as no project state apps instance is passed
to `emit_post_migrate_signal`, and `update_contenttypes` uses the global
`apps` instance.
I don't really know enough about the recent developments regarding apps
management in Django, so I'm not sure what should be done to solve this
issue. Thanks in advance if somebody has the time to look at it.
''Context: I'm testing a reusable django app ([https://bitbucket.org/tkhyn
/django-gm2m django-gm2m]) which test suite includes migration tests for a
number of mini test apps with different models (see
[https://bitbucket.org/tkhyn/django-
gm2m/src/63af327624a45c5aed110546d3d705a6b1bf7c2b/tests/?at=release
here]). Whenever I switch to another mini-app, the database needs not only
to be flushed but also re-migrated and the contenttypes need to be updated
!''
--
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:1>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.10
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:2>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.10
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Old description:
> Hi,
>
> I noticed that when one runs `migrate` without arguments (implicitly
> migrating all apps), the `apps` instance passed to
> `emit_post_migrate_signal`
> (https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L221
> [here]) contains the migrated apps' configurations only in the first run.
>
> On subsequent runs, the `plan` list
> ([https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L136
> here]) is empty and `post_migrate_apps` is set to `pre_migrate_apps`
> ([https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L197
> here]), i.e. the unmigrated apps.
>
> For example, when calling `migrate` with the `contenttypes` app enabled,
> this means that after the first run `update_contenttypes`
> ([https://github.com/django/django/blob/1.10/django/contrib/contenttypes/management.py#L89
> here]) is passed an `apps` instance with `contenttypes` missing. The
> consequence is that `update_contenttypes` can not go further than
> [https://github.com/django/django/blob/1.10/django/contrib/contenttypes/management.py#L102
> this line].
>
> To temporarily fix the issue and correctly trigger the
> `post_migrate_signal` so that the content types are properly updated each
> time `migrate` is called (and not only the first time), I need to run
> `flush` just after `migrate`. I don't think it should not be needed to
> run `flush` simply to correctly trigger the `post_migrate_signal` that
> will update the content types.
>
> It worked fine in Django 1.9.9 as no project state apps instance is
> passed to `emit_post_migrate_signal`, and `update_contenttypes` uses the
> global `apps` instance.
>
> I don't really know enough about the recent developments regarding apps
> management in Django, so I'm not sure what should be done to solve this
> issue. Thanks in advance if somebody has the time to look at it.
>
> ''Context: I'm testing a reusable django app
> ([https://bitbucket.org/tkhyn/django-gm2m django-gm2m]) which test suite
> includes migration tests for a number of mini test apps with different
> models (see [https://bitbucket.org/tkhyn/django-
> gm2m/src/63af327624a45c5aed110546d3d705a6b1bf7c2b/tests/?at=release
> here]). Whenever I switch to another mini-app, the database needs not
> only to be flushed but also re-migrated and the contenttypes need to be
> updated !''
New description:
Hi,
I noticed that when one runs `migrate` without arguments (implicitly
migrating all apps), the `apps` instance passed to
`emit_post_migrate_signal`
[https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L221
here]) contains the migrated apps' configurations only in the first run.
On subsequent runs, the `plan` list
([https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L136
here]) is empty and `post_migrate_apps` is set to `pre_migrate_apps`
([https://github.com/django/django/blob/1.10/django/core/management/commands/migrate.py#L197
here]), i.e. the unmigrated apps.
For example, when calling `migrate` with the `contenttypes` app enabled,
this means that after the first run `update_contenttypes`
([https://github.com/django/django/blob/1.10/django/contrib/contenttypes/management.py#L89
here]) is passed an `apps` instance with `contenttypes` missing. The
consequence is that `update_contenttypes` can not go further than
[https://github.com/django/django/blob/1.10/django/contrib/contenttypes/management.py#L102
this line].
To temporarily fix the issue and correctly trigger the
`post_migrate_signal` so that the content types are properly updated each
time `migrate` is called (and not only the first time), I need to run
`flush` just after `migrate`. I don't think it should not be needed to run
`flush` simply to correctly trigger the `post_migrate_signal` that will
update the content types.
It worked fine in Django 1.9.9 as no project state apps instance is passed
to `emit_post_migrate_signal`, and `update_contenttypes` uses the global
`apps` instance.
I don't really know enough about the recent developments regarding apps
management in Django, so I'm not sure what should be done to solve this
issue. Thanks in advance if somebody has the time to look at it.
''Context: I'm testing a reusable django app ([https://bitbucket.org/tkhyn
/django-gm2m django-gm2m]) which test suite includes migration tests for a
number of mini test apps with different models (see
[https://bitbucket.org/tkhyn/django-
gm2m/src/63af327624a45c5aed110546d3d705a6b1bf7c2b/tests/?at=release
here]). Whenever I switch to another mini-app, the database needs not only
to be flushed but also re-migrated and the contenttypes need to be updated
!''
--
Comment (by timgraham):
I'm not sure I understand entirely. I think the idea is that if no
migrations are applied, then content types shouldn't need to be updated.
How can we reproduce the issue to look into it? Run the test suite for
your app?
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:3>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.10
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by charettes):
* cc: charettes (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:4>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.10
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by tkhyn):
Thanks for the reply,
You could try and run the test suite for the app (I haven't pushed the
changes to fix all the Django 2.0 warnings though so the suite will fail),
or more simply use a minimal project in an environment with Django 1.10
installed:
{{{
hg clone https://bitbucket.org/tkhyn/dj10_syncdb
cd dj10_syncdb
hg update a7614832e38a7f8f21cf474dc328c295a9d8fc3c
}}}
In this state, the sample project has one unmigrated app `app` with one
model `MyModel`. Lets migrate.
{{{
manage.py migrate
}}}
The django_content_type table now looks like:
|| id || app_label || model ||
|| 1 || admin || logentry ||
|| 2 || auth || permission ||
|| 3 || auth || user ||
|| 4 || auth || group ||
|| 5 || contenttypes || contenttype ||
|| 6 || sessions || session ||
|| 7 || app || mymodel ||
... and everything is ok so far. Let's add a new model to our unmigrated
app.
{{{
hg update dfd8067076c1ceb7d9a1ae48ddc27c0dfd64044c
manage.py migrate
}}}
The django_content_type table has not changed. There should be a new line
in django_content_type (app_label = app and model = mysecondmodel), and
it's not there because the ``update_contenttypes`` cannot proceed further
than line 102 when `migrate` is called a second time.
A possible solution would be to transform `app` into a migrated app (I
haven't checked it though), but given that it worked ok in django 1.9.9 I
see this as a regression.
Feel free to let me know if you require additional info on this.
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:5>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.10
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by tkhyn):
I should have added that using `migrate --run-syncdb` does not change the
outcome (which can be expected when looking at the `migrate` management
command implementation as it has no influence on the apps instance that is
passed to the `post_migrate` signal)
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:6>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.10
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by charettes):
* severity: Normal => Release blocker
* stage: Unreviewed => Accepted
Comment:
Hi tkhyn,
I can't look at this in details in the next few days but
`update_contenttypes` should definitively be running for applications
without migrations.
The fact applications with no migrations are present in `apps` on the
first `migrate` run but not on subsequent ones look like a bug to me.
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:7>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.10
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by tkhyn):
Thanks @charettes.
> The fact applications with no migrations are present in `apps` on the
first `migrate` run but not on subsequent ones look like a bug to me.
Note that on subsequent runs it's not `app`'config that is missing in the
`pre_migrate_apps` ' app configs, but actually `contenttypes`'config.
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:8>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.10
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by charettes):
Thanks for the follow up @tkhyn,
In this case I suspect the bug lies `MigrationExecutor`, it might be
related to #26647.
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:9>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: charettes
Type: Bug | Status: assigned
Component: Migrations | Version: 1.10
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by charettes):
* owner: nobody => charettes
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:10>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: charettes
Type: Bug | Status: assigned
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by charettes):
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/7133 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:11>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: charettes
Type: Bug | Status: assigned
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:12>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: charettes
Component: Migrations | Version: 1.10
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Simon Charette <charette.s@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"d1757d8df486b689172d2584ded52fad916bcc33" d1757d8d]:
{{{
#!CommitTicketReference repository=""
revision="d1757d8df486b689172d2584ded52fad916bcc33"
Fixed #27044 -- Included already applied migration changes in the post-
migrate state when the execution plan is empty.
Refs #24100.
Thanks tkhyn for the report and Tim for the review.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:13>
Django
even when no migration has been applied to them
Reporter: tkhyn | Owner: charettes
Type: Bug | Status: closed
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Simon Charette <charette.s@…>):
In [changeset:"1c60765d6307cf5bb659d9157a990d7410108424" 1c60765d]:
{{{
#!CommitTicketReference repository=""
revision="1c60765d6307cf5bb659d9157a990d7410108424"
[1.10.x] Fixed #27044 -- Included already applied migration changes in the
post-migrate state when the execution plan is empty.
Refs #24100.
Thanks tkhyn for the report and Tim for the review.
Backport of d1757d8df486b689172d2584ded52fad916bcc33 from master
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/27044#comment:14>