Best way to use a 'all' QuerySet as a dict with id first

[aRkadeFR]

Hey!

I'm having a hard time trying to reduce the number of SQL queries on a view.

Basically, I'm fetching all my User, with User.objects.all(), and save this
queryset as AllUser.

Then every time I need to get the user with the id = X , I'm calling a function
'get_user_from_id', that iterate over the AllUser queryset variable, and when
it finds the id, returns it.

I considerably reduced the number of SQL queries, but I would like to know if
you think of a better way?

Thank you

[Alejandro Varas G.]

Hi,

You should use User.objects.get(id=X)

Best

> --
> You received this message because you are subscribed to the Google Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/20140926182939.GA26744%40rkade-thinkpad.
> For more options, visit https://groups.google.com/d/optout.

[James Brewer]

Do you have a single list of the IDs you want? If so, you can do something like `User.objects.get(id__in=user_ids)`. This will fetch all Users whose `id` is in some list `user_ids`.

Does that help?

Happy hacking!

James

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAL60nj%2BBtqs9CXu8drOiWPoJ7aUkAKjLqXKCxmR_TMTeOOH%3DGw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

James Brewer

jamesbrewer.io

[aRkadeFR]

@James Brewer:
If I change my code, I can have this user_ids list.
btw, it's filter and not get if you're searching multiple objects (the
user_ids).

@Alejandro Varas G.:
That doesn't change the fact that 'User.objects.get(id=X)' will hit the
database everytime.

Right now, the problem is solved, by creating the AllUserSet, but I think my
code is pretty ugly, and it seems strange to me that the ORM can't handle that
built-in, by caching the .all() or .filter(id__in=users_ids)

The ideally solution would be something like:

AllUser = User.objects.all()

for i in user_id:
AllUser.get(id=i)

But this code hits the database every time the 'get' is called.

> > <https://groups.google.com/d/msgid/django-users/CAL60nj%2BBtqs9CXu8drOiWPoJ7aUkAKjLqXKCxmR_TMTeOOH%3DGw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> > .

> >
> > For more options, visit https://groups.google.com/d/optout.
> >
>
>
>
> --
> James Brewer
> jamesbrewer.io
>

> --
> You received this message because you are subscribed to the Google Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.

> To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAKj8pVpqBgpFDQ0BruHXLWe2q1CRURMEejokhMKpD5bfs9hGGA%40mail.gmail.com.

[tkdchen]

On Saturday, September 27, 2014 9:06:58 PM UTC+8, aRkadeFR wrote:

@James Brewer:
If I change my code, I can have this user_ids list.
btw, it's filter and not get if you're searching multiple objects (the
user_ids).

@Alejandro Varas G.:
That doesn't change the fact that 'User.objects.get(id=X)' will hit the
database everytime.

Right now, the problem is solved, by creating the AllUserSet, but I think my
code is pretty ugly, and it seems strange to me that the ORM can't handle that
built-in, by caching the .all() or .filter(id__in=users_ids)

Django actually caches all objects fetched when it evaluates .all or .filter method. The problem you are facing is that how to reuse the cached objects efficiently to avoid hitting database with unnecessary SQL queries. James gave you a good solution. Based on his solution, you don't need to call the `get' method each time getting user by id. Just iterate users object and find the right one by comparing id.

 

The ideally solution would be something like:

    AllUser = User.objects.all()

    for i in user_id:
        AllUser.get(id=i)

This piece of code would be changed to

AllUser = User.objects.all()

for i in user_id:

    for user in AllUser:

        if user.pk == i:

            print 'find the user'

or, even you may construct AllUser to dict to simplify the search

AllUser = dict((user.pk, user) for user in User.objects.all()) # Here, hit db, only once

for i in user_id:

    user = AllUser.get(i, None) # search by id in memory

    if user is not None:

        print 'find the user'

Hope, this could help you. 

[aRkadeFR]

Thanks for the answer, I just wondered if there was a built-in solution in
Django.

> > <javascript:>>

> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > You should use User.objects.get(id=X)
> > > >
> > > > Best

> > > > El 26/09/2014 15:28, "aRkadeFR" <con...@arkade.info <javascript:>>

> > > > an email to django-users...@googlegroups.com <javascript:>.

> > > > > To post to this group, send email to django...@googlegroups.com

> > <javascript:>.

> > > > > Visit this group at http://groups.google.com/group/django-users.
> > > > > To view this discussion on the web visit
> > > >
> > https://groups.google.com/d/msgid/django-users/20140926182939.GA26744%40rkade-thinkpad
> > > > .
> > > > > For more options, visit https://groups.google.com/d/optout.
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Django users" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > an

> > > > email to django-users...@googlegroups.com <javascript:>.

> > > > To post to this group, send email to django...@googlegroups.com

> > <javascript:>.

> > > > Visit this group at http://groups.google.com/group/django-users.
> > > > To view this discussion on the web visit
> > > >
> > https://groups.google.com/d/msgid/django-users/CAL60nj%2BBtqs9CXu8drOiWPoJ7aUkAKjLqXKCxmR_TMTeOOH%3DGw%40mail.gmail.com
> > > > <
> > https://groups.google.com/d/msgid/django-users/CAL60nj%2BBtqs9CXu8drOiWPoJ7aUkAKjLqXKCxmR_TMTeOOH%3DGw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >
> > > > .
> > > >
> > > > For more options, visit https://groups.google.com/d/optout.
> > > >
> > >
> > >
> > >
> > > --
> > > James Brewer
> > > jamesbrewer.io
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > Groups "Django users" group.
> > > To unsubscribe from this group and stop receiving emails from it, send

> > an email to django-users...@googlegroups.com <javascript:>.

> > > To post to this group, send email to django...@googlegroups.com

> > <javascript:>.

> > > Visit this group at http://groups.google.com/group/django-users.
> > > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/django-users/CAKj8pVpqBgpFDQ0BruHXLWe2q1CRURMEejokhMKpD5bfs9hGGA%40mail.gmail.com.
> >
> > > For more options, visit https://groups.google.com/d/optout.
> >
>

> --
> You received this message because you are subscribed to the Google Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.

> To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/2d9a29de-9326-4ddc-a716-df9383b2aa32%40googlegroups.com.