[Django] #31665: Auto-migrations fail on postgres when resizing Charfield and setting default value longer than old constraint

4 views

Skip to first unread message

Django

unread,

Jun 5, 2020, 10:14:42 AM6/5/20

to django-...@googlegroups.com

#31665: Auto-migrations fail on postgres when resizing Charfield and setting
default value longer than old constraint
-------------------------------------+-------------------------------------
Reporter: | Owner: nobody
shadytradesman |
Type: Bug | Status: new
Component: | Version: 2.2
Migrations | Keywords: charfield resize
Severity: Normal | default alterfield max_length
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Migration to create the field

{{{
# Generated by Django 2.2.12 on 2020-05-29 19:06

import cells.models
from django.db import migrations, models

class Migration(migrations.Migration):

dependencies = [
('cells', '0003_auto_20200529_1832'),
]

operations = [
migrations.AddField(
model_name='cell',
name='invite_link_secret_key',
field=models.CharField(default=cells.models.random_string,
max_length=7),
),
]
}}}

Migration to resize the field and add a default:
{{{
# Generated by Django 2.2.12 on 2020-05-29 23:56

import cells.models
from django.db import migrations, models

class Migration(migrations.Migration):

dependencies = [
('cells', '0004_auto_20200529_2006'),
]

operations = [
migrations.AlterField(
model_name='cell',
name='invite_link_secret_key',
field=models.CharField(default=cells.models.random_string,
max_length=64),
),
]

}}}

Here is the random string method:
{{{
def random_string():
return hashlib.sha224(bytes(random.randint(1, 99999999))).hexdigest()
}}}

This migration appeared to work on mysql when I was developing locally,
but it may be because I updated the random_string() function to provide a
string that was longer after I ran the migration. It failed when I ran it
against my prod Postgres 9.5.15 database.

--
Ticket URL: <https://code.djangoproject.com/ticket/31665>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Jun 5, 2020, 10:16:52 AM6/5/20

to django-...@googlegroups.com

#31665: Auto-migrations fail on postgres when resizing Charfield and setting
default value longer than old constraint
-------------------------------------+-------------------------------------

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by shadytradesman:

Old description:

New description:

Migration to create the field

{{{
# Generated by Django 2.2.12 on 2020-05-29 19:06

import cells.models
from django.db import migrations, models

class Migration(migrations.Migration):

dependencies = [
('cells', '0003_auto_20200529_1832'),
]

operations = [
migrations.AddField(
model_name='cell',
name='invite_link_secret_key',
field=models.CharField(default=cells.models.random_string,
max_length=7),
),
]
}}}

Migration to resize the field:

{{{
# Generated by Django 2.2.12 on 2020-05-29 23:56

import cells.models
from django.db import migrations, models

class Migration(migrations.Migration):

dependencies = [
('cells', '0004_auto_20200529_2006'),
]

operations = [
migrations.AlterField(
model_name='cell',
name='invite_link_secret_key',
field=models.CharField(default=cells.models.random_string,
max_length=64),
),
]

}}}

Here is the random string method:
{{{
def random_string():
return hashlib.sha224(bytes(random.randint(1, 99999999))).hexdigest()
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/31665#comment:1>

Django

unread,

Jun 5, 2020, 10:18:34 AM6/5/20

to django-...@googlegroups.com

#31665: Auto-migrations fail on postgres when Charfield's default value is longer
than old constraint
-------------------------------------+-------------------------------------
Reporter: shadytradesman | Owner: nobody
Type: Bug | Status: new

Component: Migrations | Version: 2.2
Severity: Normal | Resolution:
Keywords: charfield resize | Triage Stage:
default alterfield max_length | Unreviewed

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by shadytradesman:

Old description:

> Migration to create the field

>
> {{{
> # Generated by Django 2.2.12 on 2020-05-29 19:06
>
> import cells.models
> from django.db import migrations, models
>

> class Migration(migrations.Migration):
>
> dependencies = [
> ('cells', '0003_auto_20200529_1832'),
> ]
>
> operations = [
> migrations.AddField(
> model_name='cell',
> name='invite_link_secret_key',
> field=models.CharField(default=cells.models.random_string,
> max_length=7),
> ),
> ]
> }}}
>

> Migration to resize the field:

> {{{
> # Generated by Django 2.2.12 on 2020-05-29 23:56
>
> import cells.models
> from django.db import migrations, models
>

> class Migration(migrations.Migration):
>
> dependencies = [
> ('cells', '0004_auto_20200529_2006'),
> ]
>
> operations = [
> migrations.AlterField(
> model_name='cell',
> name='invite_link_secret_key',
> field=models.CharField(default=cells.models.random_string,
> max_length=64),
> ),
> ]
>
> }}}
>
> Here is the random string method:
> {{{
> def random_string():
> return hashlib.sha224(bytes(random.randint(1, 99999999))).hexdigest()
> }}}
>

> This migration appeared to work on mysql when I was developing locally,
> but it may be because I updated the random_string() function to provide a
> string that was longer after I ran the migration. It failed when I ran it
> against my prod Postgres 9.5.15 database.

New description:

Migration to create the field

{{{
# Generated by Django 2.2.12 on 2020-05-29 19:06

import cells.models
from django.db import migrations, models

class Migration(migrations.Migration):

dependencies = [
('cells', '0003_auto_20200529_1832'),
]

operations = [
migrations.AddField(
model_name='cell',
name='invite_link_secret_key',
field=models.CharField(default=cells.models.random_string,
max_length=7),
),
]
}}}

Migration to resize the field:

{{{
# Generated by Django 2.2.12 on 2020-05-29 23:56

import cells.models
from django.db import migrations, models

class Migration(migrations.Migration):

dependencies = [
('cells', '0004_auto_20200529_2006'),
]

operations = [
migrations.AlterField(
model_name='cell',
name='invite_link_secret_key',
field=models.CharField(default=cells.models.random_string,
max_length=64),
),
]

}}}

Here is the random string method:
{{{
def random_string():
return hashlib.sha224(bytes(random.randint(1, 99999999))).hexdigest()
}}}

This migration worked on mysql when I was developing locally, but it was

because I updated the random_string() function to provide a string that
was longer after I ran the migration. It failed when I ran it against my
prod Postgres 9.5.15 database.

I suggest including default_value functions in the migrations so that when
old migrations are run against new code, you don't have issues.

--
Ticket URL: <https://code.djangoproject.com/ticket/31665#comment:2>

Django

unread,

Jun 5, 2020, 10:19:20 AM6/5/20

to django-...@googlegroups.com

Component: Migrations | Version: 2.2
Severity: Normal | Resolution:
Keywords: charfield resize | Triage Stage:
default alterfield max_length | Unreviewed

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by shadytradesman:

Old description:

> Migration to create the field

>
> {{{
> # Generated by Django 2.2.12 on 2020-05-29 19:06
>
> import cells.models
> from django.db import migrations, models
>

> Migration to resize the field:

> {{{
> # Generated by Django 2.2.12 on 2020-05-29 23:56
>
> import cells.models
> from django.db import migrations, models
>

> This migration worked on mysql when I was developing locally, but it was

> because I updated the random_string() function to provide a string that
> was longer after I ran the migration. It failed when I ran it against my
> prod Postgres 9.5.15 database.
>

> I suggest including default_value functions in the migrations so that
> when old migrations are run against new code, you don't have issues.

New description:

Migration to create the field

{{{
# Generated by Django 2.2.12 on 2020-05-29 19:06

import cells.models
from django.db import migrations, models

class Migration(migrations.Migration):

dependencies = [
('cells', '0003_auto_20200529_1832'),
]

operations = [
migrations.AddField(
model_name='cell',
name='invite_link_secret_key',
field=models.CharField(default=cells.models.random_string,
max_length=7),
),
]
}}}

Migration to resize the field:

{{{
# Generated by Django 2.2.12 on 2020-05-29 23:56

import cells.models
from django.db import migrations, models

class Migration(migrations.Migration):

dependencies = [
('cells', '0004_auto_20200529_2006'),
]

operations = [
migrations.AlterField(
model_name='cell',
name='invite_link_secret_key',
field=models.CharField(default=cells.models.random_string,
max_length=64),
),
]

}}}

Here is the random string method:
{{{
def random_string():
return hashlib.sha224(bytes(random.randint(1, 99999999))).hexdigest()
}}}

This migration worked on mysql when I was developing locally, but it was

because I updated the random_string() function to provide a string that
was longer after I ran the migration. It failed when I ran it against my
prod Postgres 9.5.15 database.

I suggest including default value functions in the migrations so that when

old migrations are run against new code, you don't have issues.

--
Ticket URL: <https://code.djangoproject.com/ticket/31665#comment:3>

Django

unread,

Jun 5, 2020, 10:31:38 AM6/5/20

to django-...@googlegroups.com

Keywords: charfield resize | Triage Stage:
default alterfield max_length | Unreviewed

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* status: new => closed
* resolution: => wontfix

Comment:

> This migration worked on mysql when I was developing locally, but it was

because I updated the random_string() function to provide a string that
was longer after I ran the migration.

If you alter a function referenced by historical migration you need to
make sure the existing operations referencing the function are repointed
to a valid definition. By changing the return of `random_string` you
happened to make `0004_auto_20200529_2006` migration invalid.

In your case that means you should have copied your old version of
`random_string` that returned a string of length 7 to
`0004_auto_20200529_2006` add adjust the `AddField` operation to point to
it instead of the altered `cells.models.random_string`.