Problem migrating from South to Django migrations for Linux distributions

[Raphael Hertzog]

[ Please keep me in CC ]

Hello,

I'm one of the python-django Debian package maintainers and I have been
working on preparing the field for Django 1.7... and we have one problem
that we don't know how to handle.

Consider that Debian contains Django but also Django applications using
South. When our users will upgrade from Debian 7 to Debian 8
they will upgrade at the same time Django itself and the Django applications.

The new versions of the Django applications are likely to have unapplied
schema migrations managed by South but the system will have Django 1.7
and we have no means to let the users apply those schema changes.

I see two ways to go forward:
- either we find a way to apply South migrations with Django 1.7
- or we enhance Django migrations to be able to tag some point
in the Django migrations as equivalent to some other point in the
South migrations, that way we can recreate the supposedly-unapplied
South migrations with Django 1.7 and mark those as unneeded if all
South migrations were already applied

How can we solve this problem?

Thank you for your help!
--
Raphaël Hertzog ◈ Writer/Consultant ◈ Debian Developer

Discover the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/

[Joseph Curtin]

Normally, in development with south. If the migrations get to the point were they're out of sync. The lead Dev removed the conflicting migrations and runs the schema migration command.

As for the django 1.7 flow of migrations, I cannot comment on it, but I would go into this problem with the assumption: I should be able to create an initial migration and go from there.

One complication I see with this, is that only the django runtime knows about south which makes it near impossible to detect this in your packaging.

My question now would be, can django migrations and south migrations co-exist and can I turn off django migrations?

Cheers,
-Joseph Curtin

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/20140724202733.GA8255%40x230-buxy.home.ouaza.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Manfre]

On Thu, Jul 24, 2014 at 4:44 PM, Joseph Curtin <4...@jbcurtin.io> wrote:

My question now would be, can django migrations and south migrations co-exist and can I turn off django migrations?

No, migrations is a required feature of Django and cannot be turned off. The best you could do is generate the SQL from the migrations and manually apply to the database.

Regards,

Michael Manfre

[Andrew Godwin]

So, South migrations will not work with Django 1.7. Period.

There is no way around this; it's unfortunate that the packaging situation means that Django will get auto-upgraded as part of a distribution upgrade; I'm surprised that Debian hasn't had this with packages before? (Version upgrades that break installed but non-packaged things)

Neither of your suggested ways to go forward will work; the two history models are very different, so the tagging of positions isn't going to work, and Django 1.7 has changed substantially enough internally that porting South 1.x up to it would be a very large amount of work.

How has Debian dealt with breaking changes like this in the past? Things like Rails upgrades come to mind. This change isn't as bad - nothing will actually _break_ when you upgrade, and apps will still run, but migrate will instead fall back to non-migration mode (which won't destroy any tables, but will fail to apply any table alterations).

Also, what are the applications in particular that this will be a problem for? I'm curious to know what Django + South things Debian is shipping these days.

Andrew

[Raphael Hertzog]

Hi Andrew,

thanks for your quick answer.

On Thu, 24 Jul 2014, Andrew Godwin wrote:
> There is no way around this; it's unfortunate that the packaging situation
> means that Django will get auto-upgraded as part of a distribution upgrade;
> I'm surprised that Debian hasn't had this with packages before? (Version
> upgrades that break installed but non-packaged things)

We probably had this kind of things before and the best we can do for
non-packaged things is usally to document this in the release notes.

But for packaged things, we try usually hard to get things to just work
without any human intervention. Hence my question.

> Neither of your suggested ways to go forward will work; the two history
> models are very different, so the tagging of positions isn't going to work,
> and Django 1.7 has changed substantially enough internally that porting
> South 1.x up to it would be a very large amount of work.

OK.

> Also, what are the applications in particular that this will be a problem
> for? I'm curious to know what Django + South things Debian is shipping
> these days.

Applications that depend on South and have different upstream versions
in Debian 7 and Debian 8 are:
http://tracker.debian.org/pkg/python-django-voting
http://tracker.debian.org/pkg/python-django-threadedcomments
http://tracker.debian.org/pkg/python-django-reversion
http://tracker.debian.org/pkg/python-django-picklefield
http://tracker.debian.org/pkg/bcfg2

Given the package names, it probably means only a single end-user application.
The others are Django "extensions" for use in non-packaged applications.

And looking more closely the case of bcfg2, the package in Debian 7 does
not use South, it started using South in the version in Jessie so it
should be easy to deal with.

For the 4 others, they should provide some NEWS.Debian entry warning
users of the potential upgrade problem.

(Bccing the 5 relevant bug reports to keep a record of this)

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

[Raphael Hertzog]

Hi Brian,

On Fri, 25 Jul 2014, Brian May wrote:
> I can't help but think this might be unrealistic (?). Changes required for
> new versions of Django often break compatibility with old versions, and
> 1.4.5 is really old now. Just because many packages don't have versioned
> dependencies on Django (or a versioned dependency on a really old version)
> doesn't necessarily mean they will work with any 1.4.5. Anyone want to test
> every Django package in Debian sid against the Django in wheezy?

I agree with you that this is not realistic. A better alternate solution
is to document the process of how to execute the South migrations with
Django 1.6 in a virtualenv after having done the upgrade to Debian 8.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer