[Django] #32267: Unable to unapply a branch of migrations
Django
-----------------------------------------+------------------------
Reporter: Roman Odaisky | Owner: nobody
Type: New feature | Status: new
Component: Migrations | Version: 3.1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
1. Check out the master branch of a project into a new branch
2. Implement a feature
3. Make migrations
4. Apply the migrations
5. Merge updates from master into current branch
6. makemigrations --merge
7. Possibly repeat items 2–6 several times
8. Decide to abandon the feature, look for a way to restore the DB to what
the code in master expects
However, there’s no invocation of `manage.py migrate` that will unapply
all the migrations introduced by one branch.
For example:
{{{
master ...---0050---0051a---0052a
\ \
feature 0051b---0052b---0053m
}}}
We’re currently at the merge migration 0053m. We’d like to mark 0053m as
unapplied, perform the rollback operations of 0052b and 0051b and end up
at 0052a (at which point we can check out master and forget about the
existence of the abandoned branch and its migrations). Currently this is
only possible (other than rolling back and re-applying the master
migrations, which may not even be an option) by doing a number of `--fake`
migrations in a risk-prone manner.
There needs to be an option to run `manage.py migrate --before myapp
0051b` or something like that, a command that would rollback the specified
migration and all others that depend on it but leaving ones from parallel
branches unaffected. In other words, while `manage.py migrate myapp 1234`
ensures that 1234 is applied, the suggested command would ensure that the
specified migration is //not// applied (unapplying other migrations where
necessary, but as few as possible).
--
Ticket URL: <https://code.djangoproject.com/ticket/32267>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Roman Odaisky | Owner: nobody
Component: Migrations | Version: 3.1
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* status: new => closed
* resolution: => invalid
Comment:
> We’re currently at the merge migration 0053m. We’d like to mark 0053m as
unapplied, perform the rollback operations of 0052b and 0051b and end up
at 0052a (at which point we can check out master and forget about the
existence of the abandoned branch and its migrations). Currently this is
only possible (other than rolling back and re-applying the master
migrations, which may not even be an option) by doing a number of `--fake`
migrations in a risk-prone manner.
>
> There needs to be an option to run `manage.py migrate --before myapp
0051b` or something like that, a command that would rollback the specified
migration and all others that depend on it but leaving ones from parallel
branches unaffected. In other words, while `manage.py migrate myapp 1234`
ensures that 1234 is applied, the suggested command would ensure that the
specified migration is //not// applied (unapplying other migrations where
necessary, but as few as possible).
If you've already performed migrations from the feature branch you can
restore the previous state by
[https://docs.djangoproject.com/en/3.1/topics/migrations/#reversing-
migrations reversing migrations] to the `0050`, removing migrations from
the feature branch and applying again `0051a` and `0052a`. You can also
add new migrations that will reverse changes from `0051b`, `0052b`.
Working with migrations on feature branches is difficult and each
situation is different, I don't see a way to add an option to the
`migrate` command that can decide for you. Personally I would recommend
using a separate db for each feature that can be abandoned if not needed.
--
Ticket URL: <https://code.djangoproject.com/ticket/32267#comment:1>
Django
Reporter: Roman Odaisky | Owner: nobody
Component: Migrations | Version: 3.1
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Roman Odaisky):
Replying to [comment:1 Mariusz Felisiak]:
> Working with migrations on feature branches is difficult and each
situation is different, I don't see a way to add an option to the
`migrate` command that can decide for you. Personally I would recommend
using a separate db for each feature that can be abandoned if not needed.
Can we at the very least add something like `manage.py migrate --unapply-
one <app> <migration>` that would unapply one migration, having ensured
that no currently applied migrations depend on it? This will solve the use
case I outlined and maybe some others while not being able to corrupt the
DB state.
--
Ticket URL: <https://code.djangoproject.com/ticket/32267#comment:2>
Django
Reporter: Roman Odaisky | Owner: nobody
Component: Migrations | Version: 3.1
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Roman Odaisky):
By the way, do I understand correctly that if this code:
https://github.com/django/django/blob/stable/3.1.x/django/db/migrations/executor.py#L43
is modified so instead of the loop
{{{
for node in next_in_app:
for migration in self.loader.graph.backwards_plan(node):
}}}
it simply performs `self.loader.graph.backwards_plan(target)` it will do
exactly what I want, that is, migrate back to the specified migration and
then unapply that migration as well?
--
Ticket URL: <https://code.djangoproject.com/ticket/32267#comment:3>
Django
Reporter: Roman Odaisky | Owner: nobody
Component: Migrations | Version: 3.1
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mariusz Felisiak):
> Can we at the very least add something like `manage.py migrate
--unapply-one <app> <migration>` that would unapply one migration, having
ensured that no currently applied migrations depend on it? This will solve
the use case I outlined and maybe some others while not being able to
corrupt the DB state.
Migrations can be reversed, see
[https://docs.djangoproject.com/en/3.1/topics/migrations/#reversing-
migrations docs]. However adding an option to reverse a specific migration
doesn't sound like a good idea, we will not be able to ensure that a
database state is not corrupted. Migrations history must be continuous.
You're talking about reversing migrations in a database but without
continuous changes reflected in migrations. So you would like to drop a
feature branch, reverse its migrations and have a clear path without
migrations from the feature branch, and yes you can do this but outside of
Django migrations. The proper way do this in Django is to reverse code
changes from the feature branch and run `makemigrations` that will create
a new migration reversing db changes.
--
Ticket URL: <https://code.djangoproject.com/ticket/32267#comment:4>
Django
Reporter: Roman Odaisky | Owner: nobody
Component: Migrations | Version: 3.1
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Roman Odaisky):
Replying to [comment:4 Mariusz Felisiak]:
> > Can we at the very least add something like `manage.py migrate
--unapply-one <app> <migration>` that would unapply one migration, having
ensured that no currently applied migrations depend on it? This will solve
the use case I outlined and maybe some others while not being able to
corrupt the DB state.
>
> Migrations can be reversed, see
[https://docs.djangoproject.com/en/3.1/topics/migrations/#reversing-
migrations docs]. However adding an option to reverse a specific migration
doesn't sound like a good idea, we will not be able to ensure that a
database state is not corrupted. Migrations history must be continuous.
If we just do one check: that no children of this particular migration are
applied. Does that not allow us to reverse the migration while
guaranteeing consistency?
--
Ticket URL: <https://code.djangoproject.com/ticket/32267#comment:5>
Django
Reporter: Roman Odaisky | Owner: nobody
Component: Migrations | Version: 3.1
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Roman Odaisky):
It turned out to be rather simple to implement:
https://github.com/django/django/pull/13781
--
Ticket URL: <https://code.djangoproject.com/ticket/32267#comment:6>
Django
Reporter: Roman Odaisky | Owner: nobody
Component: Migrations | Version: 3.1
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* resolution: invalid => wontfix
Comment:
Again, you're trying to fix a really specific situation that we don't want
to support, IMO. You want to unapply migrations and then (probably) remove
reverted migrations, this is not a flow that is supported. Also, this can
corrupt a database if you will have any migrations created after a "merge"
migration (`0053m` in your example).
--
Ticket URL: <https://code.djangoproject.com/ticket/32267#comment:7>
Django
Reporter: Roman Odaisky | Owner: nobody
Component: Migrations | Version: 3.1
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mariusz Felisiak):
You can start a discussion on DevelopersMailingList if you don't agree.
--
Ticket URL: <https://code.djangoproject.com/ticket/32267#comment:8>
Django
Reporter: Roman Odaisky | Owner: nobody
Component: Migrations | Version: 3.1
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Guðlaugur Stefán Egilsson):
I believe that what is missing here is the ability to migrate back to a
point in time based on the migrations table of that particular deployment,
instead of relying exclusively on the migrations graph. Migrations are
always applied in a linear fashion and should be "unappliable" in the
exact reverse order recorded in the migrations table, regardless of merge
migrations.
So something like
{{{
python manage.py 2022-01-17T09:46:51.444
}}}
Should unapply all migrations applied after this timestamp. This would
solve this issue here.
An alternative would be to use the ID of the migration.
{{{
python manage.py 42
}}}
Would unapply migrations with ID greater than 42 in the migrations table
for all apps.
Or am I missing something?
--
Ticket URL: <https://code.djangoproject.com/ticket/32267#comment:9>
Django
Reporter: Roman Odaisky | Owner: nobody
Component: Migrations | Version: 3.1
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Roman Odaisky):
>python manage.py migrate appname 42
>Would unapply migrations with ID greater than 42
That’s exactly what would happen, and in the case shown in the ticket,
`manage.py migrate appname 0050` will unapply both 0051a and 0051b, while
what’s desired is to unapply 0051b only.
--
Ticket URL: <https://code.djangoproject.com/ticket/32267#comment:10>