On 04/07/12 22:20, Stephen McDonald wrote:
> Hi Andrew, all.
Hi Stephen.
> My questions are:
>
> - With the above change made to Mezzanine, we can no longer use the
> --auto option with the schemamigration command as Django's
> FieldDoesNotExist gets raised. So we have to create migrations by hand
> now if they don't fall under one of the other options (--add-field,
> etc). Does this seem right? How can we get --auto working again?
This is because you're doing things beyond what South can cope with - it
was never designed to handle fields that appear dynamically at runtime,
and so I'm afraid --auto just isn't going to be reliable. The only
solution on the South end would be a major refactor - this is already
underway, but nowhere near even alpha status yet.
> - We're now getting error reports from users creating new projects
> saying that our initial/early migrations in the project that were
> created a long time ago now cause errors with missing relations on these
> custom fields [6]. How can we prevent this from happening? Should we go
> back and patch migrations to use the new tweaks we've had to do?
You share this issue with Django-CMS - it's probably because you have
fields that don't confirm to the strict migration guarantee of "nothing
must ever vanish". Your only option here is to start afresh with a new
set of initial migrations and post upgrade instructions for users on how
to get to them (migrate --fake probably needs to be applied several
times). This isn't a very good solution, but the only one available
right now, alas.
> At a broader level I'm really looking for some general advice going forward:
>
> - Should we as an open source project implemented as Django apps, even
> be providing migrations in the project? This has worked well for us over
> the years, but it seems like we could possibly avoid these types of
> problems by putting the onus on users to implement south in their own
> Mezzanine projects themselves. I really don't want to do this - with a
> project as broad as a CMS, we go a long way to bridging the gap between
> new users who are used to just uploading PHP files to a server, and who
> are now looking at a whole new realm of concepts with developing and
> maintaining Django projects. Providing these migrations has been one key
> part of bridging that gap.
You should be able to, though I agree South is not perfect considering
your database wizardry. Making users provide migrations is wrong,
however - you should be able to release a new version with new
models/etc. and users should be able to upgrade to it easily using
supplied migrations.
>
> - Does all of this point to a particular change in south? Is there a
> particular version of south we could pin to prior to this change that
> would avoid this issue? I know that doesn't sound very future proof, but
> from our perspective this has worked for a long time, and now it all
> seems to have come apart at the seams.
Not really - it's just the continual march of bugfixes and tightening
constraints, along with an aging migration codebase, which brings its
own set of problems. This feeds into the next few questions.
>
> - If all of this does point to a particular change in south, what are
> the chances of it being corrected? From the conversation in the original
> issue reported (#327) it seems like it's not considered a problem and
> won't be changing. Again I really don't know enough about the
> inner-workings of south to classify it as a bug or not.
I'll combine my answer to this with the next question.
> - Finally, and I hate to ask, but in the long term, does this entire
> dynamic field approach simply conflict with how south works at its core?
> Are these two things never going to gel together cleanly as they once
> did? If not, does anyone know of any alternatives to south that might be
> better suited for our project?
>
> Thanks for your time, and again for all of the effort that has gone into
> some obviously very complicated software.
So, at some level, yes, your approach to fields does conflict with how
South works - schemas are very fixed, immovable things, and South is
designed to bash them into shape in certain ways based on the common
Django patterns. There's no migration framework I know of that will
handle your problems (at least, not one that will work across databases
nicely).
However, I'm currently in the middle of both extending Django itself and
rewriting South to incorporate the lessons and problems of the last
three years and try and considerably improve things. I can't promise
this will lead to better handling of contribute_to_class fields, but
I'll definitely keep it in mind - it would be nice to have.
Still, at the end of the day, if you want to be able to rely on
third-party tools like South, you might want to consider relying on less
magical fields. The more specialised and clever fields get, the less
likely South (or anything else) is going to be able to work out what's
going on with them. Generic Relations are kind of an unloved child
anyway within Django, which probably doesn't help - since most people
don't use them (including me), they don't get a lot of testing even in
their original form with South.
In summary, I'm working on it, but I can't promise any results. If you
want a guaranteed fix the only solution is from someone who is affected
by this and who has free time to dive in there and try to work out
what's wrong.
Andrew