Fwd: Massively improving ManyToMany caching when using in forms
László Károlyi
as discussed in https://code.djangoproject.com/ticket/31096, I have implemented a module regarding manipulating caching on Model instances loaded from QuerySets, and hot-replaced ManyToManyField.save_form_data() to cache the instance relations from a validated ModelMultipleChoiceField, keeping the newly modified ManyToMany relations cached on the validated instance when a form gets posted.
The argument for this in my case is, I have extended a ModelForm with mixins where
after saving, I log the differences between the original and
changed instance, which has the ManyToMany
relations. One of the ManyToMany
relation's __str__()
contains a lookup that spans over two other related models.
Hence, the logging of new/changed fields takes tons of DB
queries when the instance coming from the ModelForm has uncached
relations, much less their prefetched variants.
As the ticket is closed there, I propose my module here for starting a debate on a better ManyToMany (and ForeignKey)relations handling, which my module is able to do. It was said in the ticket that the module itself is complicated, but in my humble opinion understanding the code of how the Form, ModelChoiceField and ManyToMany fields are wired together, is way more complicated. So all in all, it's just a question of viewpoint.
I'll attach three modules, the one which I pasted into the
ticket and two another that will/check return relations for
traversing them in the server code. I hope to spark some debate
and maybe provide something useful for the dev community, and
maybe have these modules taken into the project.
Mind you, this all here is voluntary. I'm already using these 3 modules in my production systems with huge speed improvements, and if nothing else, I'll opensource them and keep using them that way.
Tests: since I have tests that are testing these modules wired
up in relation with my system, unfortunately I can't post them
here. Basically I test models and relations in my own project by
evaluating the results of these functions, expecting some
wired-up results.
Cheers, -- László Károlyi http://linkedin.com/in/karolyi