[Django-EVE] Aligning eve-db models to ccp dump primary keys

[Andre Miller]

Hi there,

To build upon the work that Dylan has already done to override the
auto primary key field for the models I've made a few more changes to
align the primary keys in the Django models with the primary keys on
the CCP dump tables.

But before I commit it I just want to make sure you agree with these
changes.

I've gone though all the models that are currently defined in django-
eve-db and looked up in the latest CCP dump what the primary key
definition is. I added this primary key to the model's doc-string for
future reference.

There are a couple of scenarios:

- The primary key in the CCP dump is a single column and not a foreign
key.

We want to stop Django from auto-generating the id field and use the
value from the CCP dump, so the id column is explicitly defined:

id = models.IntegerField(unique=True, primary_key=True)

- The primary key in the CCP dump is a composite primary key (more
than one column)

Django does not support composite keys, so here we let Django auto-
create a new id column as a primary key. But to keep the database
consistent, add a constraint:

class Meta:
unique_together = ('type', 'material_type')

- The primary key in the CCP dump is a single column, but it is a
foreign key.

Set the foreign key as a unique value and a primary key in the Django
model. This will stop Django from creating an 'id' field that is not
required.

class InvMetaType(models.Model):
type = models.ForeignKey(InvType,
unique=True, primary_key=True,
related_name='inventorymetatype_type_set')

If you agree with this approach, let me know and I will commit it
after I run a test import to make sure there are no failures.

--
You received this message because you are subscribed to the Google Groups "Django and EVE Online" group.
To post to this group, send email to djang...@googlegroups.com.
To unsubscribe from this group, send email to django-eve+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-eve?hl=en.

[Dylan Liverman]

I agree with 1 and 2. This may or may not botch a couple of the
importers but those are easily fixed and should complain loudly if
they hit a snag.

Issue 3 is something I've been thinking about. Relationships like
that are usually a One to One relationship which django provides a
function for. By making it a OnetoOneField you remove some of the
ambiguity when traversing the tree.

In your InvMetaType example:

t = InvType.objects.get(id="12345") # Get a random Type
m = t.inventorymetatype_type_set.all() # Returns a set even though
there can only be one InvMetaType object associated with an InvType

If you use a OnetoOneField(primary_key=True) instead of a ForeignKey
you can do this:

t = InvType.objects.get(id="12345") # Get a random Type
m = t.invmetatype # Returns the
InvMetaType object

[Andre Miller]

Thanks Dylan,

It did hit at least one snag on the importer, which I fixed as well
(it was in AgtAgentType importer, the importer used name as a key
instead of id).

I'll play around with scenario #3 a bit tomorrow, it's 1am here right
now and my head is not as clear as it should be :).

Regards,
Andre'

[Andre Miller]

Ok, I played a bit with it, and the problem would be figuring out
which relations should be one to one and which ones not. In the exampe
relation of invtype and invmetatype there are also some invtypes which
don't have a meta type (Kernite, for example).

Is it safe to commit the changes for now keeping them all foreign keys
and we then change them to OneToOne as we find which ones should be
one to one?

At the moment they're all defined as ForeignKey and making the
ForeignKey the primary key shouldn't change existing behaviour.

And changing the ForeignKey field to a OneToOne also doesn't affect
the database schema (at least not in the test in MySQL I did), so it's
a 'soft' change that doesn't require a new import to be done.

Regards,
Andre'

[Dylan Liverman]

Go ahead and commit it and we can work out the onetoone thing with
some code on the table. Especially if the schema stays the same.

For MetaType, it'll be ok that not all InvTypes have a MetaType. As
long as all MetaTypes have only one InvType.

Internally django uses a OneToOne field for inheritance. When you
inherit a model it creates a hidden OnetoOneField (kind of like how is
creates id automatically) as primary key and links it to the model its
inheriting. I had toyed with the idea of making these models use
inheritance but that could get confusing when using the CCP Dump as
reference.

While we're on the topic of relationships...

Some of the tables appear to only be a join table. In django, this
would be a ManytoManyField. For example:

dgmEffects:
-list of effects

invTypes:
-list of inventory object types

dgmTypeEffects:
-TypeID
-EffectID
-isDefault

Here, dgmTypeEffects shows what effects a invtype has available to
it. If you have a type object and want to query its effects, the code
can be rather cumbersome. If we add the following code:

dgmEffects:
types = models.ManytoManyField(invTypes, through='dgmTypeEffects')

invTypes:
effects = models.ManytoManyField(dgmEffects, through='dgmTypeEffects')

dgmTypeEffects:
type = models.ForeignKey(invTypes)
effect = models.ForeignKey(dgmEffects)
is_default = models.BooleanField(default=False)

http://docs.djangoproject.com/en/dev/topics/db/models/#extra-fields-on-many-to-many-relationships

Using the 'through' parameter will override django's automatically
created join table. This will greatly simplify queries and still
allow us to query the extra fields on the join table.

type = invTypes.objects.get(id='12345')
effects = type.effects.all()
default_effects = type.effects.filter(dgmTypeEffects__is_default=True)
low_slot_items = invTypes.objects.filter(effects__name="loPower")

The ManytoMany change would take a lot of thought and some work. But
I've been thinking a lot about it a lot and wanted to get it written
down somewhere before I forget.

Dylan