Database agnostic EnumField

[Ashley Waite]

I've been working a bit on an EnumField implementation because it'll save me a lot of future time in a project, and existing implementations seem to be fragile, non-reversible, or one-database-only. Wondering why there isn't a PEP435 based EnumField in Django itself, I didn't find many answers with a search on the mailing list.

Is this a feature that would be considered, and if so, what would the expectations on it be?

I was a bit reluctant on all the implementations I could find because they seem to reduce to these issues:
* Using an int/char instead of database enum
* Being database vendor specific
* Requiring a non-standard enum or sub-class of it
* Not allowing enum reuse
* Not migrating changes statefully (ie, injecting type declaration on connection in postgres)
* Not allowing enum changes (add/remove/rename)
* Not parametising enum values (mysql)
* Not handling data consistency on enum changes

And realised, maybe that's why it's not a standard field.

I've done a POC implementation that works for mysql and postgres, and should be able to easily generalise to work on any database via two flags (has_enum, and requires_enum_declaration) that determine how to deal with changes to it.

It addresses all of these issues, migrates, and with a little more work can handle data effects as well.

So where should I go with this from here?
https://github.com/ashleywaite/django-more/tree/master/django_enum

- Ashley

[Jani Tiainen]

Hi.

In general features that can live outside Django (doesn't require changes in Django core to be implemented) needs quite well established de facto standard in community to be included in the Django itself.

So best thing to do is to make sure that your implementation can work with all current backends and preferably with all current supported Django versions.

And get users to your enum field.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/317b5aea-b68f-467b-886d-68a49c7194c7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tom Forbes]

There is a maybe/someday ticket for this here: https://code.djangoproject.com/ticket/24342

If your interested you could work on adding this into Django through that ticket, it seems like an interesting idea.

When removing an enum value, how do you handle existing records? Are all supported databases consistent in throwing errors if existing rows are still present with a removed enumeration value?

--

[Ashley Waite]

Thanks for that link Tom, I thought I'd come across that before but forgot where I saw the discussion!
That definitely seems like the right place.

Jani, I patch in several changes to core classes in order to do this without any of those caveats, but have tried to use as few as was possible.

Specifically:
* Changes to the migration state classes to allow for state tracking of types
* Changes to schema editor to allow for parametised db_types
* Changes to migration autodetector to have operations generated and in the right places (enums before models)

Tom, to handle existing records I apply on_delete functionality. If declared on a field (optional) it provides a default behaviour for that field, but it also asks during makemigrations (see here https://github.com/ashleywaite/django-more/blob/master/django_enum/patches.py#L28 ) and then gives the option to apply any of those equivalent behaviours to existing records, including setting the value to a newly added value so you can effectively 'rename' a value within the enum.

Then during migrate these are applied, which can update records accordingly, or block migration if there's data that has protect.

Though I do need to update this to do that part via the execute() method so that it plays properly with sqlmigrate.

- Ashley

On Saturday, October 21, 2017 at 5:39:30 AM UTC+11, Tom Forbes wrote:

There is a maybe/someday ticket for this here: https://code.djangoproject.com/ticket/24342

If your interested you could work on adding this into Django through that ticket, it seems like an interesting idea.

When removing an enum value, how do you handle existing records? Are all supported databases consistent in throwing errors if existing rows are still present with a removed enumeration value?

On Fri, Oct 20, 2017 at 12:55 PM, Ashley Waite <ashley....@gmail.com> wrote:

I've been working a bit on an EnumField implementation because it'll save me a lot of future time in a project, and existing implementations seem to be fragile, non-reversible, or one-database-only. Wondering why there isn't a PEP435 based EnumField in Django itself, I didn't find many answers with a search on the mailing list.

Is this a feature that would be considered, and if so, what would the expectations on it be?

I was a bit reluctant on all the implementations I could find because they seem to reduce to these issues:
* Using an int/char instead of database enum
* Being database vendor specific
* Requiring a non-standard enum or sub-class of it
* Not allowing enum reuse
* Not migrating changes statefully (ie, injecting type declaration on connection in postgres)
* Not allowing enum changes (add/remove/rename)
* Not parametising enum values (mysql)
* Not handling data consistency on enum changes

And realised, maybe that's why it's not a standard field.

I've done a POC implementation that works for mysql and postgres, and should be able to easily generalise to work on any database via two flags (has_enum, and requires_enum_declaration) that determine how to deal with changes to it.

It addresses all of these issues, migrates, and with a little more work can handle data effects as well.

So where should I go with this from here?
https://github.com/ashleywaite/django-more/tree/master/django_enum

- Ashley

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.