Zero Downtime Field Type Changes

[Kyle Mulka]

Hi there,

Wondering if there's any tool out there to make a zero downtime field type change easier. I want to change a django model field from one type to another, and the format of the field in the database is slightly different.

I've come up with a process I think will work, but it will be rather tedious if done manually. Wondering if there's a more automated approach or other tools out there already. I'd like to come up with a general solution for use in the future, but right now trying to tackle converting UUIDField from django-extensions to the built in django version.

https://github.com/django-extensions/django-extensions/issues/647

I was thinking about maybe using a custom field that supports both types, but not exactly sure how to write a custom field that would do that.

If it helps, the manual process I'm thinking about kind of goes as follows:

# existing model:

class Model:

field = OldField()

# add field_old with OldField type

# when updating or adding rows start writing to both fields

# copy data from field to field_old

class Model:

field = OldField()

field_old = OldField()

# stop using field, only use field_old

# remove field

class Model:

field_old = OldField()

# create field with new type

# when updating or adding rows start writing to both fields in their respective formats

# copy data from field_old in old format to field in new format

class Model:

field = NewField()

field_old = OldField()

# stop using field_old

# remove field_old

class Model:

field = NewField()

Thanks for your help!

--

Kyle Mulka

http://www.kylemulka.com

[Mike Dewhirst]

On 13/03/2019 2:40 pm, 'Kyle Mulka' via Django users wrote:
> Hi there,
>
> Wondering if there's any tool out there to make a zero downtime field
> type change easier. I want to change a django model field from one
> type to another, and the format of the field in the database is
> slightly different.
>
> I've come up with a process I think will work, but it will be rather
> tedious if done manually. Wondering if there's a more automated
> approach or other tools out there already. I'd like to come up with a
> general solution for use in the future, but right now trying to tackle
> converting UUIDField from django-extensions to the built in django
> version.
> https://github.com/django-extensions/django-extensions/issues/647
>
> I was thinking about maybe using a custom field that supports both
> types, but not exactly sure how to write a custom field that would do
> that.

I think you do need to write the second field and during the migration
use a ...

        migrations.RunPython(forwards_func)

... to populate the new field from the old one.

It should all happen in a single transaction - at least it would with
PostgreSQL - and no-one would see anything until the transaction is
committed. On completion of the migration reload the web server to start
using the new model. There will be a gap before reloading when the old
software is still using the old field.

When removing the old field which can be done straight away do the same
thing and translate the data again to pick up those few records which
might have been exposed during the time before the webserver was reloaded.

That's what I would do.

Mike

> --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users...@googlegroups.com
> <mailto:django-users...@googlegroups.com>.
> To post to this group, send email to django...@googlegroups.com
> <mailto:django...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/b16ffdee-74d6-4ba2-ac1f-b5bd9cf1e40b%40googlegroups.com
> <https://groups.google.com/d/msgid/django-users/b16ffdee-74d6-4ba2-ac1f-b5bd9cf1e40b%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[Markus Holtermann]

Hey Kyle, that sounds about right. It's a variation of the "Migration Recipe" I came up with a while ago: https://markusholtermann.eu/2016/04/django-migrations-recipe-2/ . My question would be if you really need the field of the new type to have the exact same name as the old one. If you can live or deal with the new type using a different name it's much easier as you don't need to migrate data twice.

/Markus

> --
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.