It seems like multi-table sub-classed models (not abstract) don't
honor the "pk" being specified in fixtures. For example, I have an
Event class that is sub-classed by a Race model. The Event class is in
an app called "calendar" and the Race model is in an app called
"regattas". I dumpdata regattas, reset both apps, and loaddata the
regattas fixture. It loads properly but all of the PKs are now
sequential, meaning a lot of other models are now pointing at the
wrong regatta entry. There were regattas with IDs 14, 16, and 18, but
are now 1, 2, and 3. I have confirmed that the fixture contains the
old 14, 16, and 18 primary keys.
The problem is the keys seem to be compacted rather than being loaded
from the fixture. The keys are not always sequential due to deletes.
So the problem doesn't exhibit itself until you delete, dumpdata, then
try to re-load.
Does anyone have any good way to get around this problem? It's
somewhat of a show-stopper for us.
I've been looking through the source to try to see if there is any
indicator as to whether this is the intended behavior. If nobody
speaks up today, I'll file a bug, as it would appear to be.
I've submitted a bug about this since I haven't been able to turn up
any kind of response. In the mean-time, I've had to rig something up
with generic relations to serve the same purpose. It's a really nasty
kludge, I think, which is a shame. I can't believe this hasn't cropped
up with anyone else yet, it seems like a really glaring problem,
especially for those who use fixtures.
On Mon, Jun 9, 2008 at 12:09 AM, Greg Taylor <squishywaf...@gmail.com> wrote:
> I've submitted a bug about this since I haven't been able to turn up > any kind of response. In the mean-time, I've had to rig something up > with generic relations to serve the same purpose. It's a really nasty > kludge, I think, which is a shame. I can't believe this hasn't cropped > up with anyone else yet, it seems like a really glaring problem, > especially for those who use fixtures.
Hi Greg,
Apologies for not responding sooner, even if just to let you know that you're not howling at the moon. I'm not ignoring you - I just don't have anything to add at this point.
Yes, there is some work required on serialization to support some of the additions to QS-RF. Inherited models are one problematic area; models with non-primary OneToOneFields are another (although the problems are probably related). As soon as I've finished this email, I'm going to take a look at these problems and see what I can do about a fix.
Much appreciated, Russ! I was not sure whether I was doing something
wrong or not, this is good to know this is being looked at. I tinkered
with a fix for a while but ultimately fell flat on my back for lack of
experience with Django internals. Let me know if I can assist with
testing or provide more details.
Thanks again,
Greg
On Jun 8, 9:39 pm, "Russell Keith-Magee" <freakboy3...@gmail.com>
wrote:
> On Mon, Jun 9, 2008 at 12:09 AM, Greg Taylor <squishywaf...@gmail.com> wrote:
> > I've submitted a bug about this since I haven't been able to turn up
> > any kind of response. In the mean-time, I've had to rig something up
> > with generic relations to serve the same purpose. It's a really nasty
> > kludge, I think, which is a shame. I can't believe this hasn't cropped
> > up with anyone else yet, it seems like a really glaring problem,
> > especially for those who use fixtures.
> Hi Greg,
> Apologies for not responding sooner, even if just to let you know that
> you're not howling at the moon. I'm not ignoring you - I just don't
> have anything to add at this point.
> Yes, there is some work required on serialization to support some of
> the additions to QS-RF. Inherited models are one problematic area;
> models with non-primary OneToOneFields are another (although the
> problems are probably related). As soon as I've finished this email,
> I'm going to take a look at these problems and see what I can do about
> a fix.
On Mon, Jun 9, 2008 at 8:06 PM, Greg Taylor <squishywaf...@gmail.com> wrote:
> Much appreciated, Russ! I was not sure whether I was doing something > wrong or not, this is good to know this is being looked at. I tinkered > with a fix for a while but ultimately fell flat on my back for lack of > experience with Django internals. Let me know if I can assist with > testing or provide more details.
I think I have it pretty much under control. I'm just finessing some test cases at the moment. It turns out there are actually a couple of related problems - too much data was being serialized, primary keys were missing a few critical annotations, and the save process was a little too enthusiastic about creating new parent instances. However, I think I've got a solution now; a little more testing, and there should be something in trunk. Watch this space. :-)
> On Mon, Jun 9, 2008 at 8:06 PM, Greg Taylor <squishywaf...@gmail.com> wrote:
> > Much appreciated, Russ! I was not sure whether I was doing something
> > wrong or not, this is good to know this is being looked at. I tinkered
> > with a fix for a while but ultimately fell flat on my back for lack of
> > experience with Django internals. Let me know if I can assist with
> > testing or provide more details.
> I think I have it pretty much under control. I'm just finessing some
> test cases at the moment. It turns out there are actually a couple of
> related problems - too much data was being serialized, primary keys
> were missing a few critical annotations, and the save process was a
> little too enthusiastic about creating new parent instances. However,
> I think I've got a solution now; a little more testing, and there
> should be something in trunk. Watch this space. :-)
<freakboy3...@gmail.com> wrote: > On Mon, Jun 9, 2008 at 8:06 PM, Greg Taylor <squishywaf...@gmail.com> wrote:
>> Much appreciated, Russ! I was not sure whether I was doing something >> wrong or not, this is good to know this is being looked at. I tinkered >> with a fix for a while but ultimately fell flat on my back for lack of >> experience with Django internals. Let me know if I can assist with >> testing or provide more details.
> I think I have it pretty much under control. I'm just finessing some > test cases at the moment. It turns out there are actually a couple of > related problems - too much data was being serialized, primary keys > were missing a few critical annotations, and the save process was a > little too enthusiastic about creating new parent instances. However, > I think I've got a solution now; a little more testing, and there > should be something in trunk. Watch this space. :-)
OK; committed as [7600]. Let me know if you continue to have difficulties with this.
Wonderful, I'll be able to test this pretty extensively tonight.
Since I don't think it's documented either way, is there a particular
procedure I should abide by when dumping/reloading fixtures for the
parent and child models? In my case, the parent is in one app, and the
child is another. Do I dumpdata for both of those apps and load them
in a certain order, or just dump for the child class that inherits
access to all of the fields and re-load that fixture?
Thanks again, this will get my project back on track!
On Jun 9, 10:06 am, "Russell Keith-Magee" <freakboy3...@gmail.com>
wrote:
> On Mon, Jun 9, 2008 at 8:12 PM, Russell Keith-Magee
> <freakboy3...@gmail.com> wrote:
> > On Mon, Jun 9, 2008 at 8:06 PM, Greg Taylor <squishywaf...@gmail.com> wrote:
> >> Much appreciated, Russ! I was not sure whether I was doing something
> >> wrong or not, this is good to know this is being looked at. I tinkered
> >> with a fix for a while but ultimately fell flat on my back for lack of
> >> experience with Django internals. Let me know if I can assist with
> >> testing or provide more details.
> > I think I have it pretty much under control. I'm just finessing some
> > test cases at the moment. It turns out there are actually a couple of
> > related problems - too much data was being serialized, primary keys
> > were missing a few critical annotations, and the save process was a
> > little too enthusiastic about creating new parent instances. However,
> > I think I've got a solution now; a little more testing, and there
> > should be something in trunk. Watch this space. :-)
> OK; committed as [7600]. Let me know if you continue to have
> difficulties with this.
On Mon, Jun 9, 2008 at 10:09 PM, Greg Taylor <squishywaf...@gmail.com> wrote:
> Wonderful, I'll be able to test this pretty extensively tonight.
> Since I don't think it's documented either way, is there a particular > procedure I should abide by when dumping/reloading fixtures for the > parent and child models? In my case, the parent is in one app, and the > child is another. Do I dumpdata for both of those apps and load them > in a certain order, or just dump for the child class that inherits > access to all of the fields and re-load that fixture?
I've just added some docs about this [1]. You will need to dump both parent class and child class. If they're in separate applications, that means dumping both applications (or parts of both applications, as appropriate).
When fixtures are reloaded, all the fixtures you load in a single loaddata command are loaded inside a single transaction. Referential integrity (when it's supported by your database) isn't checked until the end of the transaction, so the order of creation internal to the transaction doesn't matter. As a result, all the following:
There is, of course, once caveat to this - if you're using MySQL with InnoDB tables, all bets are off. InnoDB has referential integrity, but it doesn't defer to end of transaction, so creation order becomes important. This is a known bug, but one that is impossible to solve in a generic fashion at our end - we need MySQL to fix their referential integrity implementation. MyISAM tables aren't affected - they don't have any referential integrity, so no checks are ever performed.
> On Mon, Jun 9, 2008 at 10:09 PM, Greg Taylor <squishywaf...@gmail.com> wrote:
> > Wonderful, I'll be able to test this pretty extensively tonight.
> > Since I don't think it's documented either way, is there a particular
> > procedure I should abide by when dumping/reloading fixtures for the
> > parent and child models? In my case, the parent is in one app, and the
> > child is another. Do I dumpdata for both of those apps and load them
> > in a certain order, or just dump for the child class that inherits
> > access to all of the fields and re-load that fixture?
> I've just added some docs about this [1]. You will need to dump both
> parent class and child class. If they're in separate applications,
> that means dumping both applications (or parts of both applications,
> as appropriate).
> When fixtures are reloaded, all the fixtures you load in a single
> loaddata command are loaded inside a single transaction. Referential
> integrity (when it's supported by your database) isn't checked until
> the end of the transaction, so the order of creation internal to the
> transaction doesn't matter. As a result, all the following:
> There is, of course, once caveat to this - if you're using MySQL with
> InnoDB tables, all bets are off. InnoDB has referential integrity, but
> it doesn't defer to end of transaction, so creation order becomes
> important. This is a known bug, but one that is impossible to solve in
> a generic fashion at our end - we need MySQL to fix their referential
> integrity implementation. MyISAM tables aren't affected - they don't
> have any referential integrity, so no checks are ever performed.
