Automatic-manipulator, very slow..... design-decision/bug/misunderstanding?

[Gábor Farkas]

hi,

yesterday i was bitten by something in the manipulator-behaviour, which
i did not really expect to work that way...and i'm now not sure if i
should submit a bugreport, or it should behave like that. also, it's
quite possible that i understood this wrong, if that's the case please
correct me.

let's say you have a model called Owner, that references using a
foreignkey a Thing.

=====================
class Thing(Model):
name = CharField(maxlength=500)
=====================

=====================
class Owner(Model):
name = CharField(maxlength=500)
thing = ForeignKey(Thing)
=====================

now let's say you have a LOT of Things... like 50000.

now, when you call Owner.AddManipulator(), he's going to fetch the WHOLE
Thing-table. just to construct the AddManipulator.

(well, i REALLY hope that's not the case, but from the db-queries it
seems so)

i understand that if you want to build later a select-box from this
data, then you need all the Thing objects, but in my case that will
never happen.

and to validate it's input i think it's absolutely unnecessary to load
in the whole table.

to build the error-message, yes, there it might need it. but only when
he really has to build the error message.

so, should i report this as a bug, or is this behavior intentional?

also, while we're at it... is there a workaround?
i tried to experiment with the "follow" parameter, to have the
manipulator ignore the thing_id field, and seems to help in the
ChangeManipulator case, but in the AddManipulator case it does not help,
because then i cannot save the object.

is there a way to tell the manipulator to take some additional data for
the object-save?

thanks,
gabor

[Michael Radziej]

Gábor Farkas wrote:
> =====================
> class Owner(Model):
> name = CharField(maxlength=500)
> thing = ForeignKey(Thing)
> =====================
>
> now let's say you have a LOT of Things... like 50000.
>
> now, when you call Owner.AddManipulator(), he's going to fetch the WHOLE
> Thing-table. just to construct the AddManipulator.

Change it to
ForeignKey(Thing, raw_id_admin=True)
and Django won't try to construct the select box.

Yes, I find this quirky, too ;-) I hope that the manipulator
replacement will deal with this in a more sane way.

Michael

[Gábor Farkas]

manymanymanymanymany thanks :)

phew, and it's also documented in the model_api docs :-(. again
something that i could have found.

btw. it would be nice probably to mention in the docs that it also works
outside of the admin.

but anyway, manymany thanks.

gabor

[Ivan Sagalaev]

Gábor Farkas wrote:
> now, when you call Owner.AddManipulator(), he's going to fetch the WHOLE
> Thing-table. just to construct the AddManipulator.

I've seen that in one of my projects and fixed it by filtering out
unneeded selected with 'follow'. And it seems to me like it should be
this way.

[Gábor Farkas]

hi,

is there a way to filter with the "follow"?
for me it seems you can enable/disable certain fields...

and that does not help with the AddManipulator, because then those
fields are not even saved...

gabor

[Ivan Sagalaev]

Gábor Farkas wrote:
> is there a way to filter with the "follow"?
> for me it seems you can enable/disable certain fields...
>
> and that does not help with the AddManipulator, because then those
> fields are not even saved...

I mean when you're creating a manipulator you can list fields that you
don't want it to touch:

manipulator = Owner.AddManipulator(follow={'thing': False})

Then it will know that it doesn't need to construct a <select> for
Things at all.

Then of course you won't be able to save this field with the
manipulator. So if you need it then I think raw_id_admin is the way to go.

[Jeremy Dunck]

On 8/31/06, Ivan Sagalaev <Man...@softwaremaniacs.org> wrote:
> I mean when you're creating a manipulator you can list fields that you
> don't want it to touch:
>
> manipulator = Owner.AddManipulator(follow={'thing': False})

Thanks; I don't see that documented. Probably because manipulators
will go away...

[Gary Wilson]

I'm not sure if it's up to date, but there is mention of the follow
here:
http://code.djangoproject.com/wiki/NewAdminChanges#Manipulatorsandinlineediting

[Jeff Forcier]

Yea, 'follow' was a new feature of sorts put into the new-admin branch
way back when, by (sadly absent these days) Robert Wittams. I don't
believe it was ever fully documented even after new-admin was merged to
trunk (which was around or before 0.91, I believe).

Regards,
Jeff

[Malcolm Tredinnick]

Hey Gabor,

On Thu, 2006-08-31 at 09:16 +0200, Gábor Farkas wrote:
[...]

> let's say you have a model called Owner, that references using a
> foreignkey a Thing.
>
> =====================
> class Thing(Model):
> name = CharField(maxlength=500)
> =====================
>
>
> =====================
> class Owner(Model):
> name = CharField(maxlength=500)
> thing = ForeignKey(Thing)
> =====================
>
> now let's say you have a LOT of Things... like 50000.
>
> now, when you call Owner.AddManipulator(), he's going to fetch the WHOLE
> Thing-table. just to construct the AddManipulator.
>
> (well, i REALLY hope that's not the case, but from the db-queries it
> seems so)

It is so. It doesn't need to be: we could make it a lazy iterator,
although that is not a 10 minute change (I tried it a few months ago
when helping somebody on IRC and it took more than 10 minutes and still
wasn't right and I've never gone back to fix it completely).

Like a few other people noticing warts with manipulators, I've kind of
been punting this hoping that the manipulator/form changes will make it
go away or we can fix it in the new version (probably the latter). Since
that now appears to be moving forwards, I'm going to stay the course
there, since there are so many other things to do.

Regards,
Malcolm