[Django] #35151: makemigrations hangs forever on complex apps/models
Django
---------------------------------------+---------------------------------
Reporter: Seth Syberg | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 5.0
Severity: Normal | Keywords: migrations hang
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
---------------------------------------+---------------------------------
I have a relatively complex django/wagtail application with a large number
of apps, models, and migrations. Because this is a private project where
we have full control over how migrations roll out, every 6 months or so we
delete all migrations and reproduce them into a single file per app (I
believe this is a fairly common practice and significantly improves
development setup and CD times for us). This has worked for years until
recently... Now when we run makemigrations it hangs forever after
producing the initial migration for the first 6 or 7 apps.
I'm certain there is something about our setup causing a circular
dependency issues or the like, which is certainly not django's problem,
but the hanging/lack of error/lack of output does feel like a legitimate
bug to me as we have no way to debug what is happening.
I understand this is a particularly nebulous / difficult to reproduce bug
so I'm not sure what next steps could be. Perhaps this is a feature
request for much more explicit output during makemigrations to facilitate
debugging.
--
Ticket URL: <https://code.djangoproject.com/ticket/35151>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Seth Syberg | Owner: nobody
Component: Migrations | Version: 5.0
Severity: Normal | Resolution: needsinfo
Keywords: migrations hang | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Simon Charette):
* status: new => closed
* resolution: => needsinfo
Comment:
Hello Seth,
Unfortunately as you've acknowledged yourself there is little Django can
do to help with the level of details of you provided; if you can't
identify whether the migration framework or your project is to blame while
having to the source of Django and your project there is very little
triagers can do to help without a way to reproduce.
I would suggest adding breakpoints to the source code of the
`makemigration` command
[https://github.com/django/django/blob/06d0a1bd56a9899c351ca047a05813e8dd6a4e17/django/db/migrations/autodetector.py#L42
as well as] the `Autodetector` and try to narrow down the origin of the
issue.
TicketClosingReasons/UseSupportChannels might come handy in your debugging
adventure.
--
Ticket URL: <https://code.djangoproject.com/ticket/35151#comment:1>
Django
---------------------------------+--------------------------------------
Reporter: Seth Syberg | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 5.0
Severity: Normal | Resolution: needsinfo
Keywords: migrations hang | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------------------------
Hmm I think it goes without saying that the migration framework is to
blame right? Even if there's something funky in my models I would expect
the migration framework to tell me something about it and make it possible
to debug. Suggesting that I should set breakpoints directly in the code
you claim is not to blame smells a bit funny to me.
--
Ticket URL: <https://code.djangoproject.com/ticket/35151#comment:2>
Django
---------------------------------+--------------------------------------
Reporter: Seth Syberg | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 5.0
Severity: Normal | Resolution: needsinfo
Keywords: migrations hang | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------------------------
Replying to [comment:2 Seth Syberg]:
> Hmm I think it goes without saying that the migration framework is to
blame right? Even if there's something funky in my models I would expect
the migration framework to tell me something about it and make it possible
to debug. Suggesting that I should set breakpoints directly in the code
you claim is not to blame smells a bit funny to me.
Hello Seth!
Django is a community driven project so the framework needs everyone's
help for improving and progressing. That includes reporting tickets
providing as much information as possible, and for cases like this, a
failing test case or a reproducer are crucial for us to action it.
Do you think you could try to provide a minimal Django test project
showcasing the hanging when running `makemigrations`?
--
Ticket URL: <https://code.djangoproject.com/ticket/35151#comment:3>
Django
---------------------------------+--------------------------------------
Reporter: Seth Syberg | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 5.0
Severity: Normal | Resolution: needsinfo
Keywords: migrations hang | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------------------------
One point of clarification on the `Autodetector` suggestion - I'm running
`makemigrations` on an empty database and no `initial` migrations at all.
It's as if I just coded all of my models at once and ran `makemigrations`
for the first time. In theory this should always work right? Especially
given that this is live production code and there exists a set of
migrations that can be run successfully to create this database state.
I'm open to trying to produce a minimal reproduction, but I have no idea
where to even begin. This only cropped up after we'd created 7+ apps with
100s of models. I think if I knew how to create a minimal reproduction,
I'd also know what to comment out to create an `initial` migration and
wouldn't be submitting this ticket at all.
--
Ticket URL: <https://code.djangoproject.com/ticket/35151#comment:4>
Django
---------------------------------+--------------------------------------
Reporter: Seth Syberg | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 5.0
Severity: Normal | Resolution: needsinfo
Keywords: migrations hang | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------------------------
Comment (by Seth Syberg):
FWIW I was able to unblock myself here, but something still smells very
odd to me, I'll add some details for posterity/anyone who ends up here.
Maybe this is better suited as a feature request to (1) better handle
custom user models and (2) make debug output more helpful.
As I googled around about this I found lots of info about custom user
models/auth causing issues (including this now 8 year old open ticket:
https://code.djangoproject.com/ticket/25313) so I had a hunch it was
related to this. What finally got me over the hump though was killing the
hung makemigrations (ctrl-c) and attempting to rerun makemigrations gave a
little more information (on the 2nd run it oddly did not hang but errored
with a circular dependency error). This resulted in a flurry changes and
moving add fields around and deleting some other field references that I
was getting errors on and then running makemigrations again, and
eventually I did end up in a stable state. I went from 200+ migration
files across 7 apps to about 12 migration files total and spinning up now
is much much faster.
--
Ticket URL: <https://code.djangoproject.com/ticket/35151#comment:5>