SyntaxError exception when using ForeignKey

[Ryan Crumley]

All,

Recently when using startmigration I started getting SyntaxError
exceptions (example included at end of email). This appears to happen
whenever startmigration is creating a ForeignKey for a model which
references a model in a different application. FK's to models in the
same application do not cause this problem. I can temporarily comment
out the offending FK's, run startmigration, edit the script to include
the FK's I commented out and the migration will run successfully.

Any idea what might be causing this? Prior to a week or two ago I was
able to perform this work flow without issue. Note that all other
aspects of the application are working successfully (I am not getting
syntax errors when running the application for instance).

Thanks,

Ryan


$python manage.py startmigration myapp --initial
+ Added model 'myapp.MyClass'
Traceback (most recent call last):
File "manage.py", line 32, in <module>
execute_manager(settings)
File "/Library/Python/2.5/site-packages/django/core/management/
__init__.py", line 340, in execute_manager
utility.execute()
File "/Library/Python/2.5/site-packages/django/core/management/
__init__.py", line 295, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Library/Python/2.5/site-packages/django/core/management/
base.py", line 192, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Library/Python/2.5/site-packages/django/core/management/
base.py", line 219, in execute
output = self.handle(*args, **options)
File "/Users/ryan/prj/apps/external_apps/south/management/commands/
startmigration.py", line 265, in handle
fields = modelsparser.get_model_fields(model)
File "/Users/ryan/prj/apps/external_apps/south/modelsparser.py",
line 309, in get_model_fields
tree = get_model_tree(model)
File "/Users/ryan/prj/apps/external_apps/south/modelsparser.py",
line 168, in get_model_tree
tree = STTree(parser.suite(source).totuple())
File "<string>", line 1
from django.db import models
^
SyntaxError: invalid syntax

[Mark Ellul]

Hi Ryan,

I have run into this issue as well, with the latest South code.

However, I isolated it to the issue where the app was not in my
project folder structure. By creating a test app and using a
foreignkey from the app I want to create migrations for, to the test
app, and South could indeed startmigrations for my main app.

On further testing, I copied the model I am importing from my
Reference Application (http://code.google.com/p/django-regions/source/
browse/) into my application I am trying to create migrations, and it
worked fine.

I also tried to create Migrations for the django-regions models, and
it fails with the same syntax errors.

So, a work around is to copy the model you need into your app, which
is far from ideal, but it works.

If anyone from the south team can give me some other pointers of where
I should look for the issue. It would be appreciated.

Basically in summary, copying the referenced model into the app where
we want works, however when I try to run a startmigration on the
regions app directly, or with my application referencing the regions
app, I get the Syntax Error.

Any help would be appreciated.. Below is the code which works inside
of my apps model, but fails when doing a startmigration directly on
it.

Regards

Mark

*****************************

from django.utils.translation import ugettext_lazy as _
from django import forms
from django.utils.text import capfirst
from django.core import urlresolvers


class Country(models.Model):
"""
International Organization for Standardization (ISO) 3166-1
Country list

* ``iso`` = ISO 3166-1 alpha-2
* ``name`` = Official country names used by the ISO 3166/MA in
capital letters
* ``printable_name`` = Printable country names for in-text use
* ``iso3`` = ISO 3166-1 alpha-3
* ``numcode`` = ISO 3166-1 numeric

Note::
This model is fixed to the database table 'country' to be more
general.
Change ``db_table`` if this cause conflicts with your database
layout.
Or comment out the line for default django behaviour.

"""
iso = models.CharField(_('ISO alpha-2'), max_length=2,
primary_key=True)
name = models.CharField(_('Official name (CAPS)'), max_length=128)
printable_name = models.CharField(_('Country name'),
max_length=128)
iso3 = models.CharField(_('ISO alpha-3'), max_length=3, null=True)
numcode = models.PositiveSmallIntegerField(_('ISO numeric'),
null=True)

region_module = models.CharField(max_length=255, blank=True)
region_choices = models.CharField(max_length=255, blank=True)

active = models.BooleanField(default=True)

def has_regions(self):
return (self.region_module != '' and self.region_choices !=
'') or len(self.regionentry_set.all())

class Meta:
db_table = 'country'
verbose_name = _('Country')
verbose_name_plural = _('Countries')
ordering = ('name',)

def __unicode__(self):
return self.printable_name

def get_ajax_url(self):
return urlresolvers.reverse('region-codes',
kwargs={'country': self.iso})

[Andrew Godwin]

Also, Mark, in relation to this issue, it looks more to be a problem
with the Python parser module not being quite the same as the real one;
can you attach the models.py file of the model at the other end of the
ForeignKey? It looks like that's where it's choking.

Andrew

[Mark Ellul]

Hi Andrew,

Do you mean my model in my App?

If so I am posting it below.

Basically, as far as I know its a python 2.5 in the buildout that is used (2.5.2 to be exact)

Whats perplexing is that when I copied the model from the django-regions app into my own app the startmigrations --initial worked, however when I tried to do it on the django-regions app it failed with the Syntax error.

As you will see below, my model is complex.... However with the Country Model class copied into my models.py in my app, south works well, but trying to do the import of the model across apps it fails.

You surely know the innards better than i, however my gut feeling is that something in the django-regions app model fails, thats why the Country model works directly in my Application, but not in its own Application.

Please let me know if I can do any more, South is Django's missing piece of the Puzzle, thanks for all of your help

Regards

Mark


##################################

models.py in my application
##################################

from django.db import models
from model import ExpanderField
from django.contrib.auth.models import User


class CommonFields:
    create_date = models.DateTimeField()
    created_by = models.ForeignKey(User, related_name = 'created_by_set')
    last_modified_by = models.ForeignKey(User, related_name = 'last_modified_by_set')
    active = models.BooleanField(default=True)

class Merchant(CommonFields, models.Model):
    id = models.IntegerField(primary_key=True)
    name = models.CharField(max_length=100)
    country = models.ForeignKey(Country)
    affiliate = models.ForeignKey(Affiliate)
    common = ExpanderField(CommonFields)
   
    def __unicode__(self):
        return unicode("%s_%s"%(self.country.iso3, self.name))
   


###################################

model.py in my utils for the ExpanderField
##################################

mport types
from copy import deepcopy
from django.db import models

class ExpanderField(object):
    """
    This type of field can be used to include some common fields and methods in models.
    It take a class with those common things as parameter in the constructor and will
    add all fields in that class to models classes in which a field of
    this class type is declared. If the class containing common fields
    have fields of type: ForeignKey and ManyToManyField, then the related_name attribute
    of those fields will be renamed to a value of the form: model_name_original_value .
    Ej: if related_name = child_set and model name is MenuItem, it will be renamed to
    MenuItem_child_set. To add methods of the common class you must inherit from it.
    An importan thing to note here is that you must inherit first from the common class,
    order matters.

    Use case:
    class ContentManager(models.Manager):
        pass
       
    class CommonFields:
        pub_date = models.DateTimeField()
        created_by = models.ForeignKey(User, related_name = 'created_by_set')
        last_modified_by = models.ForeignKey(User, related_name = 'last_modified_by_set')

        objects = ContentManager()

        def save(self):
            #do something
            pass
           
    class NewsItem(CommonFields, models.Model):
        title = models.CharField(maxlength = 100)
        body = models.CharField(maxlength = 200)
        common = ExpanderField(CommonFields)

    this will create a class of this form:

    class NewsItem(models.Model):
        title = models.CharField(maxlength = 100)
        body = models.CharField(maxlength = 200)
        pub_date = models.DateTimeField()
        created_by = models.ForeignKey(User, related_name = 'created_by_set')
        last_modified_by = models.ForeignKey(User, related_name = 'last_modified_by_set')

        objects = ContentManager()

    """
    def __init__(self, field_container_class):
        self.field_container_class = field_container_class

    def contribute_to_class(self, cls, name):
        attr_list = [attr for attr in dir(self.field_container_class)
                     if attr not in  ('__doc__', '__module__', '__class__', '__dict__')]
        container = self.field_container_class
       
        for attr in attr_list:
            clone = None
            attr_value = getattr(container, attr)
           
            if type(attr_value) != types.MethodType:
                clone = deepcopy(attr_value)
               
                if (isinstance(clone, models.ForeignKey) or
                    isinstance(clone, models.ManyToManyField)) and \
                    clone.rel.related_name is not None:
                    clone.rel.related_name = cls.__name__ + '_' + clone.rel.related_name
               
            if clone is not None:
                cls.add_to_class(attr, clone)

##################################


Mark Ellul

Blog: http://blog.catalystic.com
Link Blog: http://www.google.com/reader/public/atom/user/13607417594960986784/state/com.google/broadcast

2009/4/12 Andrew Godwin <and...@aeracode.org>

[Andrew Godwin]

Mark,

It looks like this might have been caused by
http://south.aeracode.org/ticket/122 - could you update to trunk and try
again?

> <http://self.name>))

> 2009/4/12 Andrew Godwin <and...@aeracode.org <mailto:and...@aeracode.org>>

[Mark Ellul]

Hi Andrew,

That seemed to work a treat. Now the django-regions project can be migrated, and any foreign keys in my models can be migrated as well.

Thanks for all of your hard work!

Mark Ellul

Blog: http://blog.catalystic.com
Link Blog: http://www.google.com/reader/public/atom/user/13607417594960986784/state/com.google/broadcast

2009/4/16 Andrew Godwin <and...@aeracode.org>