Converting implicit m2m through table to explicit through model with Django 1.7 migrations

[John Lucas]

Hi all,

I'm attempting to convert an implicit m2m through table to an explicit through model and want to make sure I'm not shooting myself in the foot. Any input would be greatly appreciated.

My team was on Django 1.6 until a few months ago and was still living in the South world. We had a need to take an implicit through table for a m2m field and convert it to an explicit through model so we could add some extra data to it. 

To do so we:

1) Created the new model and set db_table on the model's Meta to the existing implicit through table and setup the model's fields accordingly.

2) Created the south migration. There were no database changes since everything lined up.

3) Added the additional fields to the new through model.

4) Created another south migration.

5) Ran the migrations.

This worked well - we were able to convert the implicit through to an explicit through and add extra fields while retaining our data.

I'm now trying to do the same thing in the world of Django 1.7 migrations. The same process works, but when I run the migration created in step 2 it fails because the table already exists. To get around this, I'm experimenting with using migrations.SeparateDatabaseAndState. Since these are purely state changes, it seemed reasonable to move all of the operations created by the migration in step 2 to a list of state_operations. This works! Nothing blows up and I'm able to add the fields as needed.

However, I'm concerned this might have repercussions down the line. I've also experimented with keeping the model creation portion of the migration as state operations, but moving the field alterations on the m2ms to the top level operations list. This blows up with the following error: "you cannot alter to or from M2M fields, or add or remove through= on M2M fields." My understanding is that M2M field changes are disallowed because it's difficult for the migrations to know what to do. Accordingly, I'm worried I could be hacking myself into a corner.

So - is my approach safe? Or will there be complications down the line?

Thank you,

John

[Simon Charette]

Hi John,

Until #23034 is fixed you'll have to keep using migrations.SeparateDatabaseAndState to achieve the implicit to explicit conversion.

From what I've read of your process everything should be working fine in the long term and your approach is safe.

Could you add a mention on the ticket tracking this feature addition about the workaround you and your team used to achieve similar results? I think this could be valuable to other users until the ticket is fixed.

Cheers,
Simon

[Carsten Fuchs]

Hi John,

Am 28.09.2015 um 15:54 schrieb John Lucas:
> So - is my approach safe? Or will there be complications down the line?

I've recently had a similar / the same question. See this thread, which also contains
the solution that worked very well for me:
http://thread.gmane.org/gmane.comp.python.django.user/172038

Best regards,
Carsten

[John Lucas]

Thanks for the replies. Carsten - that link is broken for me unfortunately.

Thanks for the thumbs up Simon - appreciate the help. I'll be sure to document this once I have a moment. I have one follow up question. Both so I can see the proof in the pudding myself and because I'm generally curious about how all the magic happens, is there any way to inspect the state of the autodetector? By that I mean, I'd like to test that this is 100% equivalent to a standard migration solution (i.e. create new field, move data to new field, remove old field, create the field again, move the data back) by creating both sets of migrations (on separate branches of course) and checking the state the autodetector "thinks" it's in at the end of the migrations. Is that possible?

[Carsten Fuchs]

Am 2015-10-02 um 23:28 schrieb John Lucas:
> Thanks for the replies. Carsten - that link is broken for me unfortunately.

Sorry, this seems to be a problem with gmane.

The same thread at Google groups is here:
https://groups.google.com/d/msg/django-users/K0X8P3s3DOw/cH5PDxYjLtYJ

Best regards,
Carsten