[Django] #32827: Squashing migrations isn't very effective in complicated projects; rewrite docs to provide a preferred manual trimming process
Django
docs to provide a preferred manual trimming process
-------------------------------------+-------------------------------------
Reporter: Mike | Owner: nobody
Lissner |
Type: | Status: new
Cleanup/optimization |
Component: Database | Version: dev
layer (models, ORM) | Keywords: migrations,
Severity: Normal | squashmigration, documentation
Triage Stage: | Has patch: 1
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
I'm creating this ticket as a tracker, having already discussed this
briefly on the mailing list:
https://groups.google.com/g/django-developers/c/xpeFRpMTBZw/m/lDq78EedAwAJ
and having already created a PR for this:
https://github.com/django/django/pull/14408
My impression is that to get this back on the review queue I need this
ticket.
--
Ticket URL: <https://code.djangoproject.com/ticket/32827>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
docs to provide a preferred manual trimming process
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: migrations, | Triage Stage: Accepted
squashmigration, documentation |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Unreviewed => Accepted
Comment:
OK, happy to review a suggestion.Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/32827#comment:1>
Django
docs to provide a preferred manual trimming process
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: migrations, | Triage Stage: Accepted
squashmigration, documentation |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
Comment:
Not sure that `squashmigrations` is not the preferred option for most/many
projects. It works perfectly well for a lot of cases. 🤔
Probably _Automatically squash_ vs _Manually squash_ vs _trim_ (for me)
Comments on [https://github.com/django/django/pull/14408 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/32827#comment:2>
Django
docs to provide a preferred manual trimming process
-------------------------------------+-------------------------------------
Type: | Lissner
Cleanup/optimization | Status: assigned
Component: Documentation | Version: dev
Severity: Normal | Resolution:
Keywords: migrations, | Triage Stage: Accepted
squashmigration, documentation |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Mike Lissner
* status: new => assigned
* component: Database layer (models, ORM) => Documentation
--
Ticket URL: <https://code.djangoproject.com/ticket/32827#comment:3>
Django
docs to provide a preferred manual trimming process
-------------------------------------+-------------------------------------
Type: | Lissner
Cleanup/optimization | Status: assigned
Component: Documentation | Version: dev
Severity: Normal | Resolution:
Keywords: migrations, | Triage Stage: Accepted
squashmigration, documentation |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by David Sanders):
@Carlton & Nessita:
I agree there needs to be a bit in the docs for this; Mike has stepped
down as he doesn't have any time but I have some time:
- I agree that it should be "manually squashing" vs "automatic squashing"
- I've only tried `squashmigrations` twice but both times failed and
ended up costing me quite a few hours that my boss wasn't too happy about
:D
- The first time there was this circular dependency nightmare that I
couldn't escape
- The second I had to write some custom serialisation to deal with
django-pgtrigger & in the end it didn't work out (I don't remember all the
details as this was a few months ago)
- I decided to finally manually squash them, but using the "replaces"
feature that `squashmigrations` uses – and it worked a treat plus it
didn't take very long!
- The key thing to remember is that you *must* note down all the non-
elidable operations.
- These are the steps I used (they'll need to be tailored for general
public):
- I rm'd all my migrations
- ran makemigrations (giving it a distinct new initial name)
- used git to restore all the rm'd migrations
- copy all the filenames into the `replaces` attribute
- copied all my non-elidable operations *plus any dependencies* into
the new initial migration
- wait until all deployments have been migrated out of the "replaces
zone" then follow the rest of the instructions on the docs
Sounded like Jessamyn had some issues with missing the custom user model -
I didn't encounter this issue even though my user model was in the app I
squashed 🤔
I'd like to forward a PR if that's ok.
--
Ticket URL: <https://code.djangoproject.com/ticket/32827#comment:4>
Django
docs to provide a preferred manual trimming process
-------------------------------------+-------------------------------------
Type: | Lissner
Cleanup/optimization | Status: assigned
Component: Documentation | Version: dev
Severity: Normal | Resolution:
Keywords: migrations, | Triage Stage: Accepted
squashmigration, documentation |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Natalia Bidart):
Replying to [comment:4 David Sanders]:
> @Carlton & Nessita:
Hello! Thanks for including me.
> I agree there needs to be a bit in the docs for this; Mike has stepped
down as he doesn't have any time but I have some time:
>
> - I agree that it should be "manually squashing" vs "automatic
squashing"
I don't object to this though I don't understand exactly which scenario
would be "automatic squashing": even when running `manage.py
squashmigrations arg1 arg2`, the user has to perform manual steps
afterwards (unless I'm missing something, which can be the case!)
What I mean is that, in my experience, squashing migrations is always a
manual operation (beyond running the management command, that is). There
is more or less steps, and one may or may not use the management command.
So if you ask me I would organize the docs roughly like this:
1. Squashing migrations
You can use the provided management command ... <details here for this
option>.
If the above does not work or does not suit your needs for whatever reason
(<examples>), you could instead:
<details for option 2>
<details for option 3>
I guess my point is that at most I would use labels to distinguish the
options like: using the mgmt command, resetting, replacing.
> - I've only tried `squashmigrations` twice but both times failed and
ended up costing me quite a few hours that my boss wasn't too happy about
:D
I've did it multiple times are previous work but it required a non trivial
amount of coordination involving migration landing freezes, deployment,
squashing, migration unfreeze. We only did this when migrate was too slow
when running tests.
> - The first time there was this circular dependency nightmare that I
couldn't escape
Luckily we did not have this! But we did have a test check to ensure apps
would not create circular dependencies, so I'm sure that helped.
> - The second I had to write some custom serialisation to deal with
django-pgtrigger & in the end it didn't work out (I don't remember all the
details as this was a few months ago)
> - I decided to finally manually squash them, but using the "replaces"
feature that `squashmigrations` uses – and it worked a treat plus it
didn't take very long!
> - The key thing to remember is that you *must* note down all the non-
elidable operations.
:see_no_evil:
> - These are the steps I used (they'll need to be tailored for general
public):
> - I rm'd all my migrations
> - ran makemigrations (giving it a distinct new initial name)
> - used git to restore all the rm'd migrations
> - copy all the filenames into the `replaces` attribute
> - copied all my non-elidable operations *plus any dependencies* into
the new initial migration
> - wait until all deployments have been migrated out of the "replaces
zone" then follow the rest of the instructions on the docs
I never tried this procedure!
> Sounded like Jessamyn had some issues with missing the custom user model
- I didn't encounter this issue even though my user model was in the app I
squashed 🤔
>
> I'd like to forward a PR if that's ok.
Yes please!
--
Ticket URL: <https://code.djangoproject.com/ticket/32827#comment:5>
Django
docs to provide a preferred manual trimming process
-------------------------------------+-------------------------------------
Type: | Lissner
Cleanup/optimization | Status: assigned
Component: Documentation | Version: dev
Severity: Normal | Resolution:
Keywords: migrations, | Triage Stage: Accepted
squashmigration, documentation |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by David Sanders):
Thanks nessita :)
Also I'm guilty of posting here then cross-posting in the forum as well
where Carlton had replied: https://forum.djangoproject.com/t/doc-update-
describe-steps-for-manually-squashing-migrations/20624
--
Ticket URL: <https://code.djangoproject.com/ticket/32827#comment:6>
Django
docs to provide a preferred manual trimming process
-------------------------------------+-------------------------------------
Type: | Sanders
Cleanup/optimization | Status: assigned
Component: Documentation | Version: dev
Severity: Normal | Resolution:
Keywords: migrations, | Triage Stage: Accepted
squashmigration, documentation |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: Mike Lissner => David Sanders
--
Ticket URL: <https://code.djangoproject.com/ticket/32827#comment:7>
Django
docs to provide a preferred manual trimming process
-------------------------------------+-------------------------------------
Type: | Sanders
Cleanup/optimization | Status: assigned
Component: Documentation | Version: dev
Severity: Normal | Resolution:
Keywords: migrations, | Triage Stage: Accepted
squashmigration, documentation |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by David Sanders):
Draft PR up for feedback on progress:
https://github.com/django/django/pull/16843
--
Ticket URL: <https://code.djangoproject.com/ticket/32827#comment:8>