* easy: => 0
Comment:
#14548 was a duplicate, had some useful discussion in its comments.
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
* cc: real.human@… (added)
* version: 1.1-beta-1 => SVN
* keywords: => aggregate annotate default
* needs_docs: 0 => 1
* has_patch: 0 => 1
* ui_ux: => 0
Comment:
Just added a patch with an initial implementation for this feature. It
should work for all aggregates by wrapping the default `sql_template` in
`COALESCE(%s, %%(default)s)`, if `params` has a `default` item. There's a
basic test in there that checks the behaviour for `Avg` with and without a
default. If there's positive feedback about the implementation, I'll have
a go at adding documentation and any improvements to the implementation
and tests that are suggested.
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:4>
* cc: net147 (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:5>
Comment (by lukeplant):
There is a good case for giving `Sum` a default of zero, while leaving
`Max` and `Min` with default of `None`. Essentially, the reason is that
zero is the fixed point of the `+` operator, while max and min have no
fixed points (excluding negative infinity and positive infinity
respectively, which are problematic). Also, it makes the behaviour
analogous to the Python builtins `sum`, `min` and `max` - `sum` returns 0
for an empty list, whereas `min` and `max` throw exceptions. (We don't
want to throw exceptions here, for various obvious reasons, but I think we
should be indicating 'undefined' in ''some'' way under the same
circumstances).
If we do this, we need a note in the release notes about the backwards
incompatibility - if people were relying on Sum returning None for no data
instead of zero. (We can certainly classify the previous behaviour as a
bug, since it was out of line with our docs, and neither expected or
useful behaviour, but should still mention this). We should also have
some tests that ensure that it returns a zero of the right type for
different underlying field types.
This also brings up the possibility of whether the default for `Sum`
should act like the `start` parameter of the `sum` Python builtin. I think
it should. That would make implementation harder, though, I guess.
Alternatively we could have a separate 'start' parameter for `Sum`, which
might be clearer, and would make that a separate feature.
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:6>
* cc: cvrebert (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:7>
* cc: jpk (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:8>
Comment (by jarshwah):
I would support closing this as a duplicate of #23753. Once the linked
ticket is implemented, it'll be possible for users to construct their own
coalesce value like:
{{{
annotate(sum_field = Coalesce(Sum(...), 0))
}}}
This leaves the default behaviour alone, but provides a higher level of
customisation. It relies on the #14030 patch which will also allow adding
a "start" value, as mentioned by @lukeplant above:
{{{
annotate(sum_field = Coalesce(Sum(...), 0) + Value(5))
}}}
Would everyone be happy enough with the above use?
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:10>
Comment (by mrmachine):
I'm not totally convinced (yet) that we should give up on this when
`Coalesce()` becomes available. If we just instruct people to use
`Coalesce(Sum(...), default)` all the time, we're basically just
recreating the literal semantics of SQL itself rather than providing an
easy to use abstraction of it. Sure, it would be useful and powerful to
have `Coalesce()` as well, but we might not need to force people to use it
all the time for the common case. At the very least, a decision on closing
this ticket should probably wait until `Coalesce()` is actually
implemented.
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:11>
Comment (by jarshwah):
@mrmachine that's not a bad argument to make. What we could support is:
{{{
Sum('field', default=0)
}}}
Which would then be returned from {{{ convert_value }}} in the case where
the value is None. The argument against that is we'd be special-casing the
Sum aggregate, while leaving other aggregates to use the Coalesce method.
That's not such a big downside though.
Does anyone else want to weigh in on this? Supporting a default=0 kwarg is
a 4 line + docs change.
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:12>
Comment (by devunt):
Replying to [comment:12 jarshwah]:
> @mrmachine that's not a bad argument to make. What we could support is:
>
> {{{
> Sum('field', default=0)
> }}}
>
> Which would then be returned from {{{ convert_value }}} in the case
where the value is None. The argument against that is we'd be special-
casing the Sum aggregate, while leaving other aggregates to use the
Coalesce method. That's not such a big downside though.
>
> Does anyone else want to weigh in on this? Supporting a default=0 kwarg
is a 4 line + docs change.
+1 for this.
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:13>
Comment (by LaundroMat):
Replying to [comment:12 jarshwah]:
> @mrmachine that's not a bad argument to make. What we could support is:
>
> {{{
> Sum('field', default=0)
> }}}
>
> Which would then be returned from {{{ convert_value }}} in the case
where the value is None. The argument against that is we'd be special-
casing the Sum aggregate, while leaving other aggregates to use the
Coalesce method. That's not such a big downside though.
>
> Does anyone else want to weigh in on this? Supporting a default=0 kwarg
is a 4 line + docs change.
+1 from me too.
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:14>
Comment (by digismack):
Replying to [comment:12 jarshwah]:
> @mrmachine that's not a bad argument to make. What we could support is:
>
> {{{
> Sum('field', default=0)
> }}}
>
> Which would then be returned from {{{ convert_value }}} in the case
where the value is None. The argument against that is we'd be special-
casing the Sum aggregate, while leaving other aggregates to use the
Coalesce method. That's not such a big downside though.
>
> Does anyone else want to weigh in on this? Supporting a default=0 kwarg
is a 4 line + docs change.
I ran into a situation today where this feature would have been nice. +1
from me.
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:15>
Comment (by jarshwah):
If you are running into this use case, you can use Coalesce(Sum('field'),
0) in Django 1.8. At least until a decision is made on this ticket. If you
want to bring this up on the ML for the other ORM people to have input,
that'd be a good next step.
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:16>
Comment (by Jiyda Mint Moussa):
Replying to [comment:12 Josh Smeaton]:
> @mrmachine that's not a bad argument to make. What we could support is:
>
> {{{
> Sum('field', default=0)
> }}}
>
> Which would then be returned from {{{ convert_value }}} in the case
where the value is None. The argument against that is we'd be special-
casing the Sum aggregate, while leaving other aggregates to use the
Coalesce method. That's not such a big downside though.
>
> Does anyone else want to weigh in on this? Supporting a default=0 kwarg
is a 4 line + docs change.
+1 for me .
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:17>
* cc: Simon Charette (added)
Comment:
May I suggest the argument be named `coalesce` and be added to all
aggregates (through the `django.db.models.aggregate.Aggregate` class)?
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:18>
Comment (by Aymeric Augustin):
I'm also facing this issue. The workaround with Coalesce works, but...
```
from django.db.models import Sum
from django.db.models.functions import Coalesce
```
really?
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:19>
Comment (by Simon Charette):
Note that the new aggregate functions in `django.contrib.postgres`
[https://github.com/django/django/blob/4de8aaf7ffc91b91cbb70e9db406abe9b160575a/django/contrib/postgres/aggregates/general.py#L12-L15
silently handle this].
I still believe adding a `coalesce` sugar argument to aggregate functions
would be more appropriate than a `default` one as it would allow passing
expressions instead of simple Python value.
e.g.
{{{#!python
Sum('amount', coalesce=F('minimum_amount')
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:20>
* cc: Hans Aarne Liblik (added)
Comment:
+1 for
{{{
default
}}}
kwarg
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:21>
Comment (by myartsev):
Another +1
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:22>
Comment (by bwesen):
Aaargh, cmon, 11 years? To add a perfectly sane default of 0 for Sum? I'm
amazed this hasn't been fixed for so long :) What was wrong with
supporting default=0 at least? (yes I had the same perfectly normal
reaction of getting annoyed by seeing NULL in my API output instead of 0
for a Sum of Payments for a User).
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:23>
Comment (by felixxm):
Björn, such comments doesn't help. Patch is welcome.
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:24>
* owner: (none) => Barton Ip
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:25>
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:26>
* owner: Barton Ip => Nick Pope
* needs_better_patch: 1 => 0
* needs_docs: 1 => 0
Comment:
I have produced an implementation using `Coalesce` for all aggregates
other than `Count`.
See the new [https://github.com/django/django/pull/14026 PR].
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:27>
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:28>
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:29>
* needs_better_patch: 0 => 1
Comment:
Some tests don't work on MySQL and Oracle.
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:30>
* needs_better_patch: 1 => 0
Comment:
Replying to [comment:30 Mariusz Felisiak]:
> Some tests don't work on MySQL and Oracle.
The tests are now all passing. \o/
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:31>
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:32>
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"fee87345967b3d917b618533585076cbfa43451b" fee87345]:
{{{
#!CommitTicketReference repository=""
revision="fee87345967b3d917b618533585076cbfa43451b"
Refs #10929 -- Deprecated forced empty result value for PostgreSQL
aggregates.
This deprecates forcing a return value for ArrayAgg, JSONBAgg, and
StringAgg when there are no rows in the query. Now that we have a
``default`` argument for aggregates, we want to revert to returning the
default of ``None`` which most aggregate functions return and leave it
up to the user to decide what they want to be returned by default.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:34>
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"501a8db46595b2d5b99c1d3b1146a832f43cdf1c" 501a8db4]:
{{{
#!CommitTicketReference repository=""
revision="501a8db46595b2d5b99c1d3b1146a832f43cdf1c"
Fixed #10929 -- Added default argument to aggregates.
Thanks to Simon Charette and Adam Johnson for the reviews.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:33>
Comment (by GitHub <noreply@…>):
In [changeset:"022d29c934107c515dd6d3181945146a2077bdf0" 022d29c9]:
{{{
#!CommitTicketReference repository=""
revision="022d29c934107c515dd6d3181945146a2077bdf0"
Refs #10929 -- Allowed NowUTC SQL customization for third-party backends.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:35>
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"0db8bf3d60f7a027391ce89555bdb4a95ad0a227" 0db8bf3d]:
{{{
#!CommitTicketReference repository=""
revision="0db8bf3d60f7a027391ce89555bdb4a95ad0a227"
Refs #10929 -- Fixed aggregates crash when passing strings as defaults.
Previously strings were interpreted as F() expressions and default
crashed with AttributeError:
'F' object has no attribute 'empty_result_set_value'
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:36>
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"0be8095b254fad65b2480d677ebe6098c41bbad6" 0be8095]:
{{{
#!CommitTicketReference repository=""
revision="0be8095b254fad65b2480d677ebe6098c41bbad6"
Refs #10929 -- Stopped forcing empty result value by PostgreSQL
aggregates.
Per deprecation timeline.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:37>
Comment (by GitHub <noreply@…>):
In [changeset:"b05dfc289429dc4de990617da543af681d6638c7" b05dfc28]:
{{{
#!CommitTicketReference repository=""
revision="b05dfc289429dc4de990617da543af681d6638c7"
Refs #34381, Refs #10929 -- Fixed
postgres_tests.test_aggregates.TestGeneralAggretate.test_empty_result_set()
on PostgreSQL 14+.
Follow up to 0be8095b254fad65b2480d677ebe6098c41bbad6.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/10929#comment:38>