for choice in Choice.objects.all():
print(choice.question.question_text, ':', choice.choice_text)
for choice in Choice.objects.prefetch_related('question'):
print(choice.question.question_text, ':', choice.choice_text)
for choice in Choice.objects.select_related('question'):
print(choice.question.question_text, ':', choice.choice_text)
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d402bf30-a5af-4072-8b50-85e921f7f9af%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
for choice in Choice.objects.all():
print(choice.question.author)
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/EplZGj-ejvg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1Gaha-K7KkefJkiS3LRdXvaPPwBeuKmhQv6bJFx3dty3w%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAD-wiX3Xa%3DN-D95RPGo8%3D3kN0zunuAOw-SpYUa4g_zsk63bARQ%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFO84S5Oi21dbr4VE2FC1-duLrfT%2BOB4TOMAr2UWWyzD-LwadQ%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAD-wiX3Xa%3DN-D95RPGo8%3D3kN0zunuAOw-SpYUa4g_zsk63bARQ%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFNZOJO5LEf_i%2BqG2KFUOrbTXG-yanubzjFvC1mqU-B0GGG9ng%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAD-wiX22Fn_qyvEcnLHEsPoKyvxGsrLXiGXvP%3Dz5%2BoX9W-NnNg%40mail.gmail.com.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d402bf30-a5af-4072-8b50-85e921f7f9af%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/EplZGj-ejvg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1Gaha-K7KkefJkiS3LRdXvaPPwBeuKmhQv6bJFx3dty3w%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAD-wiX3Xa%3DN-D95RPGo8%3D3kN0zunuAOw-SpYUa4g_zsk63bARQ%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/EplZGj-ejvg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFNZOJO5LEf_i%2BqG2KFUOrbTXG-yanubzjFvC1mqU-B0GGG9ng%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAD-wiX22Fn_qyvEcnLHEsPoKyvxGsrLXiGXvP%3Dz5%2BoX9W-NnNg%40mail.gmail.com.
--Adam
I agree with Marc here that the proposed optimizations are
'magical'. I think when it comes to optimizations like these you
simply cannot know in advance whether doing extra queries is going
to a be an optimization or a pessimization. If I can come up with
a single example where it would significantly decrease performance
(either memory usage or speed) compared to the default (and I'm
sure I can), then I would be strongly opposed to it ever being
default behaviour.
Concerning implementing it as an additional QuerySet method like `auto_prefetch()` - I'm not sure what I think, I feel like it could get icky (i.e. increase our technical debt), due to the way it couples things together. I can't imagine ever wanting to use it, though, I would always prefer the manual option.
Luke
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1Gaha-K7KkefJkiS3LRdXvaPPwBeuKmhQv6bJFx3dty3w%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1Gaha-K7KkefJkiS3LRdXvaPPwBeuKmhQv6bJFx3dty3w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a5780df6-ce60-05ae-88e3-997e6bc88f5c%40cantab.net.
Adam/Gordon, I'm interested in hearing how these changes led you to discovering stale prefetches?
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/3fbababd-0324-4b14-a40e-83f72c4f945c%40googlegroups.com.
I agree with Marc here that the proposed optimizations are 'magical'. I think when it comes to optimizations like these you simply cannot know in advance whether doing extra queries is going to a be an optimization or a pessimization.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1Gaha-K7KkefJkiS3LRdXvaPPwBeuKmhQv6bJFx3dty3w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a5780df6-ce60-05ae-88e3-997e6bc88f5c%40cantab.net.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d402bf30-a5af-4072-8b50-85e921f7f9af%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1Gaha-K7KkefJkiS3LRdXvaPPwBeuKmhQv6bJFx3dty3w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/8d272787-7b8a-430b-a7d4-453981f894d6%40googlegroups.com.
I almost agree with you... But I would make the behaviour an
opt-in instead of an opt-out. I wouldn't like such a behaviour to
be inserted with a simple version update.
What I wouldn't want is to give free optimisations to non-ORM pros at the cost of creating major unexpected or unneeded loads to already well optimized code bases (for which there would be close to no gain as the prefetch already exist where needed, and the only prefetch that would be added would be unnecessary).
Making it opt-in allows:
- for power ORM users not to use it at all without doing anything. We have to keep in mind that hose SQL experts may not be the ones that do the upgrades, so lead to unexpected efficiency loss without anyone to know where it comes from. As this functionality doesn't exist yet, probably no one has ever written any unit tests to guarantee that there is no unnecessary prefetch, which is the only way a non-expert would notice the upgrade is inserting a problem.
- for developers looking for better performances to set it on,
but only in debug mode and local development to understand where
they need to prefetch related fields. And when their code base is
optimal, unset this option and remove the magic and any unwanted
side effect. This would of course need to be documented in the
pages about how to optimize and get better performances with
Django.
- for new users or when creating applications mockups and proofs of concepts, they could set it on (the tutorial would talk a bit about it) and just use it, allowing them not to care about those optimizations that they don't need yet to take care of.
- Brice
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/2f0b5932-1a38-4eaf-84aa-13960a303141%40googlegroups.com.
It won't affect experienced users. They'll read the release notes, see that this change has been implemented, and either go and delete a bunch of prefetch_related() calls, grumble a bit and turn auto-prefetch off globally or just file it away as another fact they know about the Django ORM.
I wouldn't want is to give free optimisations to non-ORM pros
It won't affect experienced users. They'll read the release notes, see that this change has been implemented, and either go and delete a bunch of prefetch_related() calls, grumble a bit and turn auto-prefetch off globally or just file it away as another fact they know about the Django ORM.This is, for me, the biggest point in favour of having it opt out. It'll barely affect anyone who is really experienced, but can provide such a large upside for others.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/3bf2f56e-8534-404b-8a70-12a657be0514%40googlegroups.com.
It won't affect experienced users. They'll read the release notes, see that this change has been implemented, and either go and delete a bunch of prefetch_related() calls, grumble a bit and turn auto-prefetch off globally or just file it away as another fact they know about the Django ORM.This is, for me, the biggest point in favour of having it opt out. It'll barely affect anyone who is really experienced, but can provide such a large upside for others.
Hi Josh,
I believe we should be optimising for the **common** use case, not expecting everyone to be experts with the ORM.
> If I can come up with a single example where it would significantly decrease performance (either memory usage or speed) compared to the default (and I'm sure I can), then I would be strongly opposed to it ever being default behaviour.
The status quo is already one where thousands of users and sites are doing the non-optimal thing because we're choosing to be conservative and have users opt-in to the optimal behaviour.
A massive complaint against Django is how easy it is for users to build in 1+N behaviour. Django is supposed to abstract the database away (so much so that we avoid SQL related terms in our queryset methods), yet one of the more fundamental concepts such as joins we expect users to know about and optimise for.
I'd be more in favour of throwing an error on non-select-related-foreign-key-access than what we're currently doing which is a query for each access.
The only options users currently have of monitoring poor behaviour is:
1. Add logging to django.db.models2. Add django-debug-toolbar3. Investigate page slow downs
Here's a bunch of ways that previously tuned queries can "go bad":
1. A models `__str__` method is updated to include a related field2. A template uses a previously unused related field3. A report uses a previously unused related field4. A ModelAdmin adds a previously unused related field
I think a better question to ask is:
- How many people have had their day/site ruined because we think auto-prefetching is too magical?- How many people would have their day/site ruined because we think auto-prefetching is the better default?
If we were introducing a new ORM, I think the above answer would be obvious given what we know of Django use today.
What I'd propose:
1. (optional) A global setting to disable autoprefetching
2. An opt out per queryset3. (optional) An opt out per Meta?4. Logging any autoprefetches - perhaps as a warning?
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/2f0b5932-1a38-4eaf-84aa-13960a303141%40googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/2f0b5932-1a38-4eaf-84aa-13960a303141%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/9d359e4d-59d6-ff81-8ca6-db3bcd36ac0c%40cantab.net.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1GEzjGFXi-mP_cL-b_QFYLOZ3bVmdhy%2BaA1OzicQHx3iA%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/EplZGj-ejvg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMwjO1GEzjGFXi-mP_cL-b_QFYLOZ3bVmdhy%2BaA1OzicQHx3iA%40mail.gmail.com.
I wouldn't want is to give free optimisations to non-ORM pros
Something like 99% of django projects are by non-ORM pros, it can be easy to forget that on a mailing list of experts.
On 15 Aug 2017, at 11:44, Gordon Wrigley <gordon....@gmail.com> wrote:I'd like to discuss automatic prefetching in querysets. Specifically automatically doing prefetch_related where needed without the user having to request it.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/EplZGj-ejvg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/2D9DCC0F-EAB0-4512-8AEC-08A694DF9074%40polytechnique.org.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/EplZGj-ejvg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/bf3e22af-9a41-4106-b986-fc136d54c4c6%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/7e2115fe-dd61-4a64-8600-f3fe62fb687c%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1uoZo7fc_py2evg-T1hvANDxU2AgAm_ehdUx%3DCE%2B2pPGSQ%40mail.gmail.com.
This could work something like the way that ForeignKey
`on_delete` works - you have options that are enumerated as
constants, but in reality they are classes that embody the
strategy to be used. We could have something similar -
`on_missing_relation=FETCH | WARN | ERROR | IGNORE ... `
This could work something like the way that ForeignKey `on_delete` works - you have options that are enumerated as constants, but in reality they are classes that embody the strategy to be used. We could have something similar - `on_missing_relation=FETCH | WARN | ERROR | IGNORE ... `
I completely agree that visibility of this problem is a major issue, and would really welcome work on improving this, especially in DEBUG mode. One option might be a method that replaces lazy loads with exceptions. This would force you to add the required `prefetch_related` or `select_related` calls. You could have:
.lazy_load(None) # exception on accessing any non-loaded FK objects
.lazy_load('my_fk1', 'my_fk2') # exceptions on accessing any unloaded FK objects apart from the named ones
.lazy_load('__any__') # cancel the above, restore default behaviour
This (or something like it) would be a different way to tackle the problem - just throwing it out. You could have a Meta option to do `.lazy_load(None)` by default for a Model. I've no idea how practical it would be to wire this all up so that is works correctly, and with nested objects etc.
I'd like to discuss automatic prefetching in querysets. Specifically automatically doing prefetch_related where needed without the user having to request it.
--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/EplZGj-ejvg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d402bf30-a5af-4072-8b50-85e921f7f9af%40googlegroups.com.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.