[Django] #34050: Generated migration file is not detected by django because of the name of newly generated migration file
Django
newly generated migration file
-----------------------------------------+----------------------------
Reporter: Bishal Gautam | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 4.1
Severity: Normal | Keywords: migrations
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+----------------------------
After a new project is created we run:
`python manage.py migrate` to sync with our auth database models.
Lets create an app called myapp, register this app on settings. Now we
create a model inside myapp as:
{{{
class MyModel(models.Model):
name = models.CharField(max_length=100)
age = models.IntegerField()
}}}
Lets run `makemigrations` and `migrate` command for the app. This will
generate a migration file with name `0001_initial.py` inside
`myapp/migrations/` folder.
Now let us create a constraints for the model as:
{{{
class MyModel(models.Model):
name = models.CharField(max_length=100)
age = models.IntegerField()
class Meta:
constraints = [
models.CheckConstraint(
check=models.Q(age__gte=1),
name="Age should not be.less.than.one."
)
]
}}}
When we added CheckConstraint for our previous model, we need to run
`makemigrations` and `migrate` to reflect the new changes on database.
Running makemigrations will generate a migration file inside
`user/migrations/` with name `0002_mymodel_age should not
be.less.than.one..py`. When we try to run migrate, django will not be able
to detect this file and NO changes will be applied to existing database.
If we run makemigrations again, the same file will be generated again. No
matter how many time we run makemigrations, the same file will be
generated again and again. The newly generated migration file is not
detected by django migrate and showmigrations command. This is because of
the name of migration file. The new migration file name contains serveral
dots and because of this name, migrate/showmigration is not able to find
the file.
There are mutiple solutions for this:
- First: Renaming the generated migration file. Remove `.` (dot)
(excluding the .py)from the newly generated migration file. This will make
sure that the file is recognized as python file by django
migrate/showmigrations command.
- Second: We can change the name of our constraint name inside the
migration as :
{{{
class Meta:
constraints = [
models.CheckConstraint(
check=models.Q(age__gte=1),
name="Age should not be less than one"
)
]
}}}
The only difference between the constraint that was above and the
constraint that is implemented now is in the value of name parameter for
models.Q(). We removed the exisiting "dot" `.` from the name. Running
makemigration will generate a migration file with name `0002_mymodel_age
should not be less than one.py`. Now running the migrate/showmigrations
command will find the new migration file. This example is also commited in
the repository[https://github.com/bisalgt/test-django-migrations]. One
can look at the commit history and checkout to proper stage as needed ,
described by commit message on the repository commit history.
- Third: I have replaced the name of the migration file in django source
code so that `.`(dot) gets replaced by some character`_`. I am also
attaching the associated pull request
[https://github.com/django/django/pull/16080].
Using third approach makes sure that no one has to go through the step
First and Second as described above. The solution would work out of the
box.
--
Ticket URL: <https://code.djangoproject.com/ticket/34050>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
newly generated migration file
Reporter: Bishal Gautam | Owner: Bishal Gautam
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage: Unreviewed
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by David Sanders):
* owner: nobody => Bishal Gautam
* status: new => assigned
* has_patch: 0 => 1
* needs_tests: 0 => 1
Comment:
Can confirm this.
It's worth highlighting is that migrations literally copies the constraint
name verbatim for **any** invalid Python module character not just dots
"." and will happily apply these (as long as there's no dots) – as long as
we're not doing any imports of migrations we should be fine.
The PR replaces the dots with underscores - that's one solution which
addresses the immediate issue but I'm just wondering whether:
a.) Any invalid character is replaced to ensure there are no more problems
like this; or
b.) Migrations (ie `showmigrations` and `migrate`) do not look skip files
with multiple dots?
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:1>
Django
newly generated migration file
-------------------------------+-----------------------------------------
Reporter: Bishal Gautam | Owner: Bishal Gautam
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
* Attachment "invalid migration file name generated-unable to create
migrations file.png" added.
Invalid character in constraint name leads to invalid file so django is
unable to create new migrations file
--
Ticket URL: <https://code.djangoproject.com/ticket/34050>
Django
newly generated migration file
-------------------------------+-----------------------------------------
Reporter: Bishal Gautam | Owner: Bishal Gautam
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
Comment (by Bishal Gautam):
Yes, as an experiment, I changed the name of constraint by putting `*` in
the file name. Django created the file name with the exact name of the
constraint but failed to create a migration file because of invalid name
of file. I have also added an attachment for this. Kindly find above.
- This didnot create a migration file and generated a error from django
core. I think it still tells the user that the invalid file name causes
this issue.
- In case of dot `.` , migration file was created everytime. Generated
migration file was getting replaced by new one. Django was not able to
find the migration file using `migrate` command. So I think this dot
character error is a bit misleading to users and needs to be replaced.
Or may be we can replace most of the frequently used invalid character if
replacing doesnot compromise the performance of project.
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:2>
Django
newly generated migration file
-------------------------------+-----------------------------------------
Reporter: Bishal Gautam | Owner: Bishal Gautam
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
Comment (by David Sanders):
Nice detective work Bishal :)
It's probably worth waiting for a member of the triage team to make a
decision on whether to replace all non-valid Python identifier chars or
special filesystem chars (or maybe even decide to do something else)
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:3>
Django
newly generated migration file
-------------------------------+-----------------------------------------
Reporter: Bishal Gautam | Owner: Bishal Gautam
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
Comment (by Bishal Gautam):
Thank you so much David.
Looking forward to hear from the triage team.
Regards,
Bishal Gautam
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:4>
Django
newly generated migration file
-------------------------------+-----------------------------------------
Reporter: Bishal Gautam | Owner: Bishal Gautam
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
* needs_better_patch: 0 => 1
* stage: Unreviewed => Accepted
Comment:
I think replacing any invalid character is the way to go.
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:5>
Django
newly generated migration file
-------------------------------+-----------------------------------------
Reporter: Bishal Gautam | Owner: Bishal Gautam
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
* cc: Adam Johnson (added)
Comment:
[https://github.com/django/django/pull/16080 PR]
Regression in fa58450a9ab8a1bdd2a5090b51b00078fd85ffa6.
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:6>
Django
newly generated migration file
-------------------------------+-----------------------------------------
Reporter: Bishal Gautam | Owner: Bishal Gautam
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
Comment (by Adam Johnson):
I’ve made an alternative PR: https://github.com/django/django/pull/16117
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:7>
Django
newly generated migration file
-------------------------------+-----------------------------------------
Reporter: Bishal Gautam | Owner: Bishal Gautam
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
Comment (by Bishal Gautam):
Looks great Adam. Thank you Team for acknowledging this Ticket. Looking
forward to contribute more in future.
Regards,
Bishal Gautam
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:8>
Django
newly generated migration file
-------------------------------+-----------------------------------------
Reporter: Bishal Gautam | Owner: Bishal Gautam
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
-------------------------------+-----------------------------------------
* needs_better_patch: 1 => 0
* needs_tests: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:9>
Django
newly generated migration file
-------------------------------+----------------------------------------
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: migrations | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+----------------------------------------
* owner: Bishal Gautam => Adam Johnson
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:10>
Django
newly generated migration file
Reporter: Bishal Gautam | Owner: Adam
| Johnson
Type: Bug | Status: assigned
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:11>
Django
newly generated migration file
-------------------------------------+-------------------------------------
Reporter: Bishal Gautam | Owner: Adam
| Johnson
Component: Migrations | Version: 4.1
Severity: Normal | Resolution: fixed
Keywords: migrations | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"cd03e8e2d698e9ccefae2f7687a0400329e6cbe6" cd03e8e]:
{{{
#!CommitTicketReference repository=""
revision="cd03e8e2d698e9ccefae2f7687a0400329e6cbe6"
Fixed #34050 -- Replaced invalid chars in migration names with '_'.
Thanks to Bishal Gautam for the report and initial implementation.
Regression in fa58450a9ab8a1bdd2a5090b51b00078fd85ffa6.
Co-Authored-By: Bishal Gautam <bis...@gmail.com>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34050#comment:12>