foreignKey + manipulator...performance problem
gabor
imagine 2 simple models like:
=====================
class Thing(Model):
name = CharField(maxlength=500)
=====================
=====================
class Owner(Model):
name = CharField(maxlength=500)
thing = ForeignKey(Thing)
=====================
now let's say you have 20000 Thing objects.
everything is ok, until you call
Owner.AddManipulator()... this will take a looooooooong time.
i think it's because somehow the AddManipulator goes through all the
20000 Thing objects. i have no idea why btw. why would he need it at
this point? i can understand him needing it when the formwrapper wants
to render the field as a selectbox, but for the rest of time...why?
i tried to do something like Owner.AddManipulator(follow={'thing':False}),
and this helps, but from that point on, of course the manipulator does
not validate the thing-field in the Owner object.
i could live with that (i can validate that one input data quite easily),
but then i cannot save that object using the manipulator...
is there some way to speed up this situation?
to somehow put 'back' that thing-field in manipulator-save..?
the other way for me is to completely bypass the manipulator, but the
object with which i work is quite complex, and i'd rather not do that.
any ideas?
gabor
Gábor Farkas
> hi,
>
> imagine 2 simple models like:
>
>
> =====================
> class Thing(Model):
> name = CharField(maxlength=500)
> =====================
>
>
> =====================
> class Owner(Model):
> name = CharField(maxlength=500)
> thing = ForeignKey(Thing)
> =====================
>
> now let's say you have 20000 Thing objects.
>
> everything is ok, until you call
>
> Owner.AddManipulator()... this will take a looooooooong time.
>
> i think it's because somehow the AddManipulator goes through all the
> 20000 Thing objects. i have no idea why btw. why would he need it at
> this point? i can understand him needing it when the formwrapper wants
> to render the field as a selectbox, but for the rest of time...why?
>
ok, to make a long story short,
you need to use the raw_id_admin = True flag in the ForeignKey,
and all will be fine.
it's also in the documentation (model_api), but what's not mentioned is
that not only works in the admin-interface, but also globally.
apologies to the django-team for assuming that they did not solve this
isse :-)
gabor
Lau...@gmail.com
the database for that table, or did you have 2000 records in memory at
runtime?
Gábor Farkas
if you don't use raw_id_admin, then (from my understanding at least),
the manipulator loads all of them into the memory.
so you have all the objects from the database-table in the memory.
gabor
josep...@gmail.com
You know, I really couldn't believe it - so I went home this evening
and made a test case to work it through. Sure enough, it did *exactly*
that... I've posted a short sample of the performance degredation graph
on my blog at
http://www.rhonabwy.com/wp/2006/08/31/a-subtle-django-performance-hit/
If anyone wants the "test case" project, it is available at
http://www.rhonabwy.com/django_perf_test.zip
-joe
gabor
>
> Gábor Farkas wrote:
>> Lau...@gmail.com wrote:
>>> What do you mean with "2000 thing objects"? Are there 2000 records in
>>> the database for that table, or did you have 2000 records in memory at
>>> runtime?
>>>
>> if you don't use raw_id_admin, then (from my understanding at least),
>> the manipulator loads all of them into the memory.
>> so you have all the objects from the database-table in the memory.
>
> You know, I really couldn't believe it - so I went home this evening
> and made a test case to work it through. Sure enough, it did *exactly*
> that... I've posted a short sample of the performance degredation graph
> on my blog at
>
>
> http://www.rhonabwy.com/wp/2006/08/31/a-subtle-django-performance-hit/
>
nice :)
i'm not sure if i agree with the idea of having raw_id_admin=True by
default (hmmm..maybe i do... i don't know), but i definitively think
that this flag should be mentioned in a more "frequently read" place
than the model_api docs... maybe in the tutorials even.
gabor
Jorge Gajon
On 8/30/06, gabor <ga...@nekomancer.net> wrote:
> everything is ok, until you call
>
> Owner.AddManipulator()... this will take a looooooooong time.
I got this same problem yesterday, however I'm short in time so I
couldn't look deeper on what to do. I ended passing
follow={'field':False} and then saving the object manually but I don't
like that and will try to do something better later.
I think this should be put on a ticket as an enhancement, the
manipulator should load the foreign data only (and just only) when you
actually put a {{ form.foreign_field }} in your template and the html
select tag needs to be rendered.
I can't write up the ticket because I need to go in 5 minutes :(
Regards,
Jorge
Jorge Gajon
> I think this should be put on a ticket as an enhancement, the
> manipulator should load the foreign data only (and just only) when you
> actually put a {{ form.foreign_field }} in your template and the html
> select tag needs to be rendered.
Or maybe just wait for the impending removal of manipulators :)
James Bennett
> Or maybe just wait for the impending removal of manipulators :)
The class django.forms.Manipulator is going away, as are classes
derived from it. The functionality they provide, however, is not going
away -- it's just going to be handled differently and more cleanly.
--
"May the forces of evil become confused on the way to your house."
-- George Carlin
Joseph Heck
-joe
Jorge Gajon
On 8/31/06, Gábor Farkas <ga...@nekomancer.net> wrote:
> you need to use the raw_id_admin = True flag in the ForeignKey,
> and all will be fine.
Great!, Thank you Gábor :)
I was having the same problem too.
Cheers,
Jorge