Idea: Add .checked(<True>/<False>) to QuerySet as alternative to .filter() w/ .first()

[Steve Jorgensen]

It is a common idiom to use `<queryset>.filter(<...>).first()` as a replacement for `.get(<...>)` when `None` is wanted instead of an exception when no match is found. That works, but wouldn't the intention be more clear if that could be written as something like

<queryset>.checked(False).get(<...>)

[Adrian Torres Justo]

A common idiom is also

```
try:
    foo = Foo.objects.get(x="foo")
except Foo.DoesNotExist:
    foo = None
```

which is pretty pythonic IMO, but I wouldn't be opposed to a keyword-only argument on `get` that returns `None` if not found

```

foo = Foo.objects.get(x="foo", strict=False)

# or
foo = Foo.objects.get(x="foo", raises=False)

```

As it stands your current proposal isn't much different from filter() then first(), IMO, the method names change but the amount of chaining is the same.

[אורי‎]

I don't like get() and catching exceptions. What I usually do is something like this:

        foo_list = Foo.objects.filter(x=x)
        if (len(foo_list) == 1):
            foo = foo_list[0]

And if I need else, I can assign None or whatever. The advantage in this way is that you don't get an exception, and you deal with all numbers of elements except 1, not only 0 (it can be 2 or 3 for example). And you don't have to catch exceptions for multiple and does-not-exist. Many times I don't need else, I just don't do anything if the number of elements received is not 1.

Uri.

אורי

u...@speedy.net

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/eb6a2bc2-b72f-49ff-9e90-12913ad876c9n%40googlegroups.com.

[Jörg Breitbart]

I somewhat second what Adrian wrote:

- try-except handling seems to be the most idiomatic way to me
- a flag altering API behavior as .get(..., raises=False) might come
handy, and to avoid naming collisions in arguments this could also be
shaped as a separate get_or_none() method - but: I somewhat doubt its
usefulness just cluttering the API further for low benefit, also nothing
stops you from wrapping the exception way into such a query helper (or a
more advanced resemblance of .get as Uri does). In fact the exception
will create a tiny runtime overhead on interpreter side, but thats
really tiny compared to the db roundtrip (thus against the whole .get +
helper runtime).

Imho a chaining .checked() method on the manager/queryset is not a good
idea, as it might create more API ambiguity - What should .checked() do,
if it is not followed by .get()?

Cheers, jerch

[Dave Gaeddert]

I'll lob in my 2 cents that I actually think `get_or_none` would be great to have, in the same way that I imagine people think `get_or_create` is useful. You can try/catch yourself in both cases (example is basically the exact same https://docs.djangoproject.com/en/4.0/ref/models/querysets/#get-or-create), but especially for people new to Django, having a method for it is a lot easier to grasp. The MultipleObjectsReturned exception is more of a modeling/uniqueness error in my experience (not sure I've ever written a catch for this...), but doing .get and knowing it may not exist is super common — much easier to have a method for that than force everyone to think through all the ramifications of using .get vs .filter (and/or .first) etc.

I'd also guess that people don't want to create a Manager if they don't have to (which I'd also consider an advanced topic).

https://stackoverflow.com/questions/3090302/how-do-i-get-the-object-if-it-exists-or-none-if-it-does-not-exist-in-django

Dave

[Jörg Breitbart]

@Dave Gaeddert
Since you mentioned it - what should a get_or_none method do about
multiple objects? Still raise? Silently "cast" to None (imho
semantically wrong)? Return the underlying queryset (or any other
"multiple objects container thingy")?

The problem with .get() is - it may end in 3 different states (tristate
- None, one object, multiple objects), where only one state is the
expected case. To me handling the other unexpected ones as exceptions
seems to be a very common pattern in python (thus also newbies should be
able to handle it properly in their code).

Now a .get_or_none() would shift the "good results" to (None, one
object) still leaving multiple objects out. Did we gain here anything?
Not if multiple objects is still within reach (eg. uniqueness cannot be
assumed from the filter conditions) - ppl would still have to write
exception handling code. Just forgetting that is like a time bomb that
might go off anytime a project aggregates more production data (welcome
to error 500 websites / dying servers).

But if .get_or_none() would handle multiple objects without raising (eg.
returning the queryset) - we haven't gained anything either, we just
surfaced the fact, that .get is only a convenient function on top of
filtered search queries. And ppl still have to deal with the tristate
result in their code.

Maybe I am overlooking something - currently I dont see, how any of that
might be helpful in terms of easier API usage.


Am 23.06.22 um 16:57 schrieb Dave Gaeddert:

> --
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-develop...@googlegroups.com

> <mailto:django-develop...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-developers/c9833cf7-6113-4405-b5e6-8de2dda1fdb6n%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/c9833cf7-6113-4405-b5e6-8de2dda1fdb6n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
netzkolchose.de UG (haftungsbeschränkt)
Geschäftsführer: Jörg Breitbart
Handelsregister: HRB 504791 Amtsgericht Jena
Steuer-Nr.: 161/115/07450
USt-IdNr.: DE268234065

[Danilov Maxim]

I am completely agree with Jörg.
Simply Mixin with own get_or_none help solve you, @Dave Gaeddert your asks.

Here we have Other big issue, which always omitted between Django developers in GET method:

Multiple objects exception without Already received objects.

Get made "len", len created one or many objects. We already loose time for that (init time * N objects).
Probably I can catch Exception and decide in Exception, what I want to do with that objects. BUT! I cannot do it. Get throws error without received objects.
If I want to get those objects, I should send again request to database. I should wait, again.

Can we add queryset with cached_objects in MultipleObjectsReturned Exeption? Yes. Definitely. Have we this yet? No



Mit freundlichen Grüßen,
DI Mag. Maxim Danilov

+43(681)207 447 76
ma...@wpsoft.at

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/18eb5df7-85fd-8e8a-3632-351d90da50d3%40netzkolchose.de.

[quest...@gmail.com]

.get_or_none() probably should allow to query only pks and unique_together fields, so there will be no third state?

[Dave Gaeddert]

> Since you mentioned it - what should a get_or_none method do about multiple objects?

Personally I think `get_or_none` would/should still raise an exception for this. Can't speak for anyone else but I have never written a catch for MultipleObjectsReturned and still wouldn't (would be interesting to find some data on this)... If I'm using `get`, then I'm trying to get one or None and returning multiple probably means I made a much bigger mistake elsewhere (ex. forgot unique on a slug field). If I thought multiple was even remotely possible, I would have used `filter` instead of `get`.

Without `get_or_none`, people come up with some many other ways to get there... I'd rather people use the "official" `get_or_none` and forget to catch MultipleObjectsReturned than use `filter().first()` (because you can do it in one line) for the same situation and not realize they made a uniqueness error because the multiple results slip by silently...

> .get_or_none() probably should allow to query only pks and unique_together fields, so there will be no third state?

This is an interesting idea that would take it a step further (assuming it's possible). Then it feels like you can help people prevent the kinds of issues I'm describing up front? Even without it (at least initially), I still think there's some value in having a `get_or_none`.

I just think it's a common enough question/problem/pattern that Django could afford to have something to make it easier. I would bet there's a huge number of "try...get...except DoesNotExist...None" chunks of code in the wild that would be one line if they had an easier way to get there.

[Jörg Breitbart]

Well from my own habits to use .get() only for unique filtering (>80% on
pks only) I also see some benefit of a .get_or_none() variant - it would
stop writing those exception handler wrappers over and over halfway
during project realization. A ready-to-go official way would reduce that
to a conditional expression from line one on.

> .get_or_none() probably should allow to query only pks and
> unique_together fields, so there will be no third state?

To begin with - thats the place I dont like .get() at all. The fact that
it might end up with multiple objects is a nuisance, but we cannot do
much about it for API compat reasons. Idea - if we want such a changed
behavior - maybe call it "unique_or_none()" to make the difference
clear? If its called .get_or_none() imho it is better to stick to .get()
behavior regarding multiple objects (still raising).

[Dave Gaeddert]

> To begin with - thats the place I dont like .get() at all. The fact that
> it might end up with multiple objects is a nuisance, but we cannot do
> much about it for API compat reasons

> ...

For what it's worth, I agree with what you're saying here too. Having a `unique_or_none` (and maybe `unique`) with that field checking could be cool, but in the interest of actually getting a change into Django, I assume `get_or_none` without that behavior is a much easier place to start.

[Dave Gaeddert]

I'm new to this... anybody know how the best way to push this forward? Or who can make a decision on whether something could/should be added to Django? I see some other tickets/discussions about basically the same thing:

https://code.djangoproject.com/ticket/17546

A lot has changed in 10+ years... seems like this could be reconsidered.

[Mariusz Felisiak]

Hi,

    Adding `get_or_none()` was discussed several times and was always rejected. This thread has a nice summary. Personally, I'm still against it.

Best,
Mariusz

[Dave Gaeddert]

Hey Mariusz, thanks for the link — I found some other threads but not that one. Would you mind saying why you're personally against it (`get_or_none` specifically)?

At least how I read that thread, there was more debate about how to add it than whether to add it at all, and then Cal sort of "gave up" by suggesting a docs patch that I'm not sure ever happened anyway. I don't want to kick off a huge thing about it, but that one was 8 years ago and, like you said, it has come up a number of times over the years (maybe that's normal). 

Thanks!

Dave

[Mariusz Felisiak]

I'm against it because it's already easily achievable and it's debatable how it should behave with many results. IMO any new method would be confusing and in the case of unique filtering `get_or_none(unique_filters)` would be an alias for `.filter(unique_filters).first()`. To me, this is duplication of an API.

There are several -0 in the previous thread (Shai, Florian) and you can count -1 from me.

Best,
Mariusz

[Adam Johnson]

I'm also against adding get_or_none(), for the same reasons. Adding a method to shortcut something that can already be done doesn't seem worth it to me.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/aeb9be96-ec03-48f9-ae97-2859b62a1df6n%40googlegroups.com.

[Dave Gaeddert]

Fair enough. To me, the `get_or_none` behavior with multiple results would still be to raise an exception (so it is just like `get` in that sense). And that’s the reason I personally don’t just see it as a shortcut for `filter().first()` — I have (as I’m sure others have) made the mistake before of thinking that I was using a unique query when that wasn’t necessarily true, so the multiple results exception has caught mistakes at runtime. I’d rather have that exception than use `filter().first()` and potentially show the wrong object to the wrong user, for example, and not figure out that I have a uniqueness/db problem. I like the fact that `get` raises those exceptions, I just think that try/except DoesNotExist/None is such a common behavior that a shortcut for that would actually be useful.

Dave

[Matthias Kestenholz]

Hi

I believe the main objection is against adding additional API to querysets. get_object_or_404 only converts DoesNotExist into a 404 exception and doesn't handle MultipleObjectsReturned.

I'm not the first to suggest adding an additional shortcut, e.g. django.shortcuts.get_object_or_none which would work similarly to get_object_or_404 (and get_list_or_404 for that matter). The existing shortcuts use functionality from several parts of Django (the ORM and the request/response handling) which get_object_or_none wouldn't do, but its behavior would be so close to what get_object_or_404 is doing that it may be alright.

OTOH even the implementation of this shortcut is so simple that I just add it to each project when I have a need for it (I had a hard time finding it because it comes up much less often than I thought it did). So I am not proposing this addition personally but I wouldn't be against it either (if that counts for anything).

Best regards,

Matthias

def get_object_or_none(klass, *args, **kwargs):
    queryset = _get_queryset(klass)
    if not hasattr(queryset, "get"):
        klass__name = (
            klass.__name__ if isinstance(klass, type) else klass.__class__.__name__
        )
        raise ValueError(
            "First argument to get_object_or_404() must be a Model, Manager, "
            "or QuerySet, not '%s'." % klass__name
        )
    try:
        return queryset.get(*args, **kwargs)
    except queryset.model.DoesNotExist:

        return None

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/dbee041d-4f80-4e61-bf0a-1d6f4e2e22e6n%40googlegroups.com.

[Dave Gaeddert]

> I'm not the first to suggest adding an additional shortcut, e.g. django.shortcuts.get_object_or_none which would work similarly to get_object_or_404

Personally I'm not a fan of it being in "shortcuts". It makes sense to me that the 404 ones are there because they are only useful in a request/response lifecycle. Seems like a good call to keep that out of the standard queryset methods. But `get_or_none` could be used anywhere, and ergonomically (?) it makes more sense to me that people (especially people new to Django) would naturally find and use it if it were right next to `get` and `get_or_create` (in docs, autocomplete, search in your codebase, etc.).

> I believe the main objection is against adding additional API to querysets.

Yeah, I hope I'm not coming off as being too pushy, but I'm just curious why that is. Mariusz or Adam, did what I said about it being more than just an alias for `filter().first()` make sense or am I missing something? (Different behavior for multiple results — on accident or otherwise)

Dave

[James Bennett]

On Thu, Jul 14, 2022 at 4:09 PM Dave Gaeddert <dave.g...@gmail.com> wrote:

Yeah, I hope I'm not coming off as being too pushy, but I'm just curious why that is. Mariusz or Adam, did what I said about it being more than just an alias for `filter().first()` make sense or am I missing something? (Different behavior for multiple results — on accident or otherwise)

Speaking for myself:

I’d be against it because we’re getting into combinatorial territory with diminishing returns. Adding this one opens the door to equally long and repetitive threads in the future asking for all the others, and with less room to push back each time.

Currently, Django covers the common cases of “if you didn’t find a match, 404” and “if there’s not exactly one result for this, throw an exception and let me catch it”. The fact that you can make other variants kinda work if you use first() or similar methods doesn’t, to me, obligate Django to also provide the full matrix of methods doing all the someone-might-want-it combinations of DoesNotExist, MultipleObjectsReturned, and None. People who want specific behaviors not covered by the built-in stuff should write their own, or somebody should make a django-all-the-get-single-object-methods package and everybody who wants one of these methods can install and use it.

[Dave Gaeddert]

I'll drop it. Someone else can chime in if they want. Just to be clear, the work you all do on Django is much appreciated :) 

Thanks,

Dave