any(), all() and empty iterable
John O'Hagan
I was getting some surprising false positives as a result of not expecting
this:
all(element in item for item in iterable)
to return True when 'iterable' is empty.
I guess it goes into hairy Boolean territory trying to decide if an element is
in an item that doesn't exist (if that's what's happening), but I would have
thought not. It seems inconsistent with the behaviour of
any(element in item for item in iterable)
which returns False when 'iterable' is empty.
Sorry if this has come up before, but 'any' and 'all' make for fruitless
googling!
Any light to be shed?
Regards,
John
Chris Rebert
This is justified in formal logic by the concept of vacuous truth. See
http://en.wikipedia.org/wiki/Vacuous_truth
all(P(x) for x in y) #Original Python statement
∀x∈y. P(x) #equivalent in formal logic
¬¬[∀x∈y. P(x)] #apply Double Negation
¬[∃x∈y. ¬P(x)] #negate Universal Quantifier
English interpretation: There does not exist a counterexample `x` in
the set `y` which fails to satisfy predicate `P`.
Comment: Indeed, in the case the set `y` is empty, there is no
counterexample to be had, and thus both the final statement and the
initial equivalent one are vacuously true. Yes, this may seem
unintuitive at first. Logic is like that sometimes.
Python equivalent to final statement: not any(not P(x) for x in y)
Comment: Likewise, in the case of a empty `y`, any() returns False, so
the code simplifies to `not False`, which obviously simplifies further
to just True.
So, if you agree with the behavior of any(), you must logically agree
with the behavior of all(). ;-)
Discrete mathematics FTW!
Cheers,
Chris
--
I have a blog:
http://blog.rebertia.com
Albert Hopkins
> Hi,
>
> I was getting some surprising false positives as a result of not expecting
> this:
>
> all(element in item for item in iterable)
>
> to return True when 'iterable' is empty.
>
> I guess it goes into hairy Boolean territory trying to decide if an element is
> in an item that doesn't exist (if that's what's happening), but I would have
> thought not. It seems inconsistent with the behaviour of
>
> any(element in item for item in iterable)
>
> which returns False when 'iterable' is empty.
>
all(iterable)
Return True if all elements of the iterable are true. Equivalent
to:
def all(iterable):
for element in iterable:
if not element:
return False
return True
any(iterable)
Return True if any element of the iterable is true. Equivalent
to:
def any(iterable):
for element in iterable:
if element:
return True
return False
> Sorry if this has come up before, but 'any' and 'all' make for fruitless
> googling!
Try Googling for:
"any site:docs.python.org"
Luis Alberto Zarrabeitia Gomez
Quoting John O'Hagan <ma...@johnohagan.com>:
> Hi,
>
> I was getting some surprising false positives as a result of not expecting
> this:
>
> all(element in item for item in iterable)
>
> to return True when 'iterable' is empty.
>
[...]
> any(element in item for item in iterable)
>
> which returns False when 'iterable' is empty.
The "any" and "all" functions for python2.5 mirror the existential[1] and
uiniversal[2] quantifiers from predicate logic. In layman terms,
"any" returns true if there exists at least one element in the collection for
which the predicate ("element in item" in your case) is true. If the collection
is empty, there are no elements, and in particular, there can be no elements
that satisfy your predicate (returns false). This function/quantifier is well
defined on an empty set.
"all" returns true if for every element in the collection, the predicate holds.
If you have an empty set, there is no way that the predicate could evaluate to
false, thus, it return true. While this may seem arbitrary at first, keep in
mind that one expects the phrase "all this values in this set are true" to mean
the same as "there are no false values in this set", i.e, all(x for x in list)
== not any(not x for x in list). So, the universal quantifier/python's "all"
function may be easier to understand as: "returns true if there is no element in
the set that for which the predicate is false".
These would be "any" and "all" implementations in pure python:
def any(elements):
for element in elements:
if element:
return True
return False
def all(elements):
for element in elements:
if not element:
return False
return True
You may also want to look at [3]
K.
P.S: Forgive my english. I'm falling asleep... I hope I made sense.
[1] http://en.wikipedia.org/wiki/Existential_quantification
[2] http://en.wikipedia.org/wiki/Universal_quantification
[3] http://en.wikipedia.org/wiki/Vacuous_truth
--
Luis Zarrabeitia
Facultad de Matemática y Computación, UH
http://profesores.matcom.uh.cu/~kyrie
--
Participe en Universidad 2010, del 8 al 12 de febrero de 2010
La Habana, Cuba
http://www.universidad2010.cu
Tim Chase
>
> all(iterable)
>
> Return True if all elements of the iterable are true. Equivalent
> to:
>
> def all(iterable):
> for element in iterable:
> return False
> return True
Then I'd say the comment is misleading. An empty list has no
item that is true (or false), yet it returns true. The comment
in the docs should read "Return False if any element of the
iterable is not true" or "Return True if all elements of the
iterable are true or if the iterable is empty."
To get the behavior the original comment describes, would seem to
require an implementation something like this:
def all(iterable):
iterable = iter(iterable)
try:
element = iterable.next()
except StopIteration:
raise UnderdefinedBecauseNoElementsToCompareToTrue
while element:
try:
element = iterable.next()
except StopIteration:
return True
return False
Tweaking the documentation seems like an easier and more
backwards compatible solution to me :)
-tkc
Paul Rubin
> Then I'd say the comment is misleading. An empty list has no item
> that is true (or false), yet it returns true.
The comment is correct. "All the items of the iterable are true"
means EXACTLY the same thing as "there are no items of the iterable
that are false". The empty list has no false items. Therefore
all(empty_list) = True is the correct behavior.
Another possible implementation:
import operator,itertools
def all(xs):
return reduce(operator.and_, itertools.imap(bool, xs), True)
Arnaud Delobelle
A contest! My entry:
def all(iterable):
return not sum(not x for x in iterable)
--
Arnaud
Tim Chase
Problem with both entries: short-circuit evaluation.
def test_me(how_many=99999999999999999):
yield False
for _ in xrange(how_many): yield True
print all(test_me())
The stdlib version wisely bails on the first False. A
particularly useful aspect when test_me() does something
time-consuming:
def test_me(times=100)
for _ in xrange(times):
yield some_long_running_process_that_usually_returns_false()
where that process may do something like slurp a web-page across
the planet, or calculate some expensive expression.
-tkc
John O'Hagan
> Tim Chase <pytho...@tim.thechases.com> writes:
> > > Return True if all elements of the iterable are
> > > true. ...
> >
> > Then I'd say the comment is misleading. An empty list has no item
> > that is true (or false), yet it returns true.
>
> The comment is correct. "All the items of the iterable are true"
> means EXACTLY the same thing as "there are no items of the iterable
> that are false". The empty list has no false items. Therefore
> all(empty_list) = True is the correct behavior.
>
[...]
By the same argument, all the items of the iterable are also False, or blue,
or anything else you care to say about them, because there aren't any. From
the Wikipedia article on vacuous truth (see Chris Rebert's reply in this
thread):
"All pink rhinoceros are carnivores."
"All pink rhinoceros are herbivores."
But you're right, by this logic the comment is correct, and although the
choice of True seems arbitrary, I've learnt something about logic today.
Regards,
John
Arnaud Delobelle
I was aware of this but I mimicked the behaviour of Paul's
implementation. It's even worse if the iterable is something like
itertools.repeat(False)
--
Arnaud
John O'Hagan
> On Sat, Apr 11, 2009 at 9:00 PM, John O'Hagan <ma...@johnohagan.com> wrote:
> >
> > I was getting some surprising false positives as a result of not
> > expecting this:
> >
> > all(element in item for item in iterable)
> >
> > to return True when 'iterable' is empty.
[...]
>
> This is justified in formal logic by the concept of vacuous truth. See
> http://en.wikipedia.org/wiki/Vacuous_truth
>
> all(P(x) for x in y) #Original Python statement
> ∀x∈y. P(x) #equivalent in formal logic
> ¬¬[∀x∈y. P(x)] #apply Double Negation
> ¬[∃x∈y. ¬P(x)] #negate Universal Quantifier
>
> English interpretation: There does not exist a counterexample `x` in
> the set `y` which fails to satisfy predicate `P`.
> Comment: Indeed, in the case the set `y` is empty, there is no
> counterexample to be had, and thus both the final statement and the
> initial equivalent one are vacuously true. Yes, this may seem
> unintuitive at first. Logic is like that sometimes.
>
> Python equivalent to final statement: not any(not P(x) for x in y)
> Comment: Likewise, in the case of a empty `y`, any() returns False, so
> the code simplifies to `not False`, which obviously simplifies further
> to just True.
>
> So, if you agree with the behavior of any(), you must logically agree
> with the behavior of all(). ;-)
I do like this reasoning, but when you're testing for the ubiquity of
something you don't expect to find it in an empty space - I was checking
lists of sets to see which possible set members were in all the sets of a
list, and found they all were: if the list contained no sets! I think that is
surprising.
Or to put it another way, if I ask someone "Amongst your books, is one of
them 'Robinson Crusoe'?", and they don't have any books, they could
answer 'yes' (or 'no' equally truthfully), but I'd rather they told me that
no, they don't have 'Robinson Crusoe'.
Still, point taken, and quibble easily solved with a little extra test for
emptiness.
Thanks,
John
Peter Otten
> Or to put it another way, if I ask someone "Amongst your books, is one of
> them 'Robinson Crusoe'?", and they don't have any books, they could
> answer 'yes' (or 'no' equally truthfully), but I'd rather they told me
> that no, they don't have 'Robinson Crusoe'.
That's why you ask "Do you have any books called 'Robinson Crusoe'?" rather
than "Are all your books called 'Robinson Crusoe'?".
The difference between logic and natural language is more striking
with "or". When you ask "Do you take the bus at nine or eleven?" you
certainly don't expect "Yes" as the answer.
Peter
Tim Chase
> than "Are all your books called 'Robinson Crusoe'?".
Mu. If I don't have any books..."Have you stopped beating all
your wives?" The question makes the presumption that I have
books (or that you've been beating your wife).
-tkc
Eduardo O. Padoan
<pytho...@tim.thechases.com> wrote:
>>> From the docs:
>>
>> all(iterable)
>> Return True if all elements of the iterable are true.
>> to:
>> def all(iterable):
>> for element in iterable:
>> if not element:
>> return False
>> return True
>
> all elements of the iterable are true or if the iterable is empty."
I didn't knew about "Vacuous True" (my fault, it seems Discrete Math
101) until reading about on this thread (thanks everyone!), and
reading on wikipedia it answered this exact question.
> To get the behavior the original comment describes, would seem to require an
> implementation something like this:
>
> def all(iterable):
> iterable = iter(iterable)
> try:
> element = iterable.next()
> except StopIteration:
> raise UnderdefinedBecauseNoElementsToCompareToTrue
> while element:
> try:
> element = iterable.next()
> except StopIteration:
> return True
> return False
>
>
> Tweaking the documentation seems like an easier and more backwards
> compatible solution to me :)
>
> -tkc
>
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
Eduardo de Oliveira Padoan
http://importskynet.blogspot.com
http://djangopeople.net/edcrypt/
"Distrust those in whom the desire to punish is strong."
-- Goethe, Nietzsche, Dostoevsky
Peter Pearson
>>>From the docs:
>>
>> all(iterable)
>>
>> Return True if all elements of the iterable are true. Equivalent
>> to:
>>
>> def all(iterable):
>> for element in iterable:
>> if not element:
>> return False
>> return True
>
> Then I'd say the comment is misleading. An empty list has no
> item that is true (or false), yet it returns true. The comment
> in the docs should read "Return False if any element of the
> iterable is not true" or "Return True if all elements of the
> iterable are true or if the iterable is empty."
How 'bout: "Return True if no element of the iterable is not true"?
--
To email me, substitute nowhere->spamcop, invalid->net.
Tim Chase
I still prefer "Return False if any element of the iterable is
not true" or "Return False if any element in the iterable is
false" because that describes exactly what the algorithm does.
Granted, anybody with a mote of Python skills can tell that from
the algorithm, but if you're going to go to the trouble of
documenting, you might as well document what it does. The code
in the docs do not check the truthiness of each element, it
checks for the falseness (not-trueness) of each element. One
might be able to defend it using logical manipulations, but since
Python strives for clarity...
-tkc
Arnaud Delobelle
In fact the doc is not just misleading, but plain wrong as the following
shows:
>>> def alwaystrue():
... while True: yield True
...
>>> all(alwaystrue())
[Python is stuck]
The iterable alwaystrue() satisfies the property "all elements of the
iterable are true", but all(alwaystrue()) does not return. Instead one
could describe all()'s behaviour as:
Return False as soon as the first false element in the iterable is
found. Otherwise return True when the iterable is exhausted.
Unfortunately it makes it much less obvious what the function is for and
the Python implementation is a clearer explanation IMHO.
So in the end, I think the doc reaches a good compromise: a short easy
to understand english description that gives a clear idea of what all()
is for, and a sample implementation for those who need/want the exact
behaviour.
--
Arnaud
Carl Banks
A previous time we had this dicussion, I pointed out that natural
language comparisons are questionable since in natural langauge you
are not confined to two answers. The valid answer to the above
question could be "I don't have any books", neither yes nor no. The
closest thing to that you can get in Python is to raise an exception.
Carl Banks
John Posner
> I still prefer "Return False if any element of the iterable is not
> because that describes exactly what the algorithm does. Granted,
> anybody with a mote of Python skills can tell that from the algorithm,
> but if you're going to go to the trouble of documenting, you might as
> well document what it does. The code in the docs do not check the
> truthiness of each element, it checks for the falseness (not-trueness)
> of each element. One might be able to defend it using logical
> manipulations, but since Python strives for clarity...
> Return False as soon as the first false element in the iterable is
> found. Otherwise return True when the iterable is exhausted.
>
> Unfortunately it makes it much less obvious what the function is for and
> the Python implementation is a clearer explanation IMHO.
>
> So in the end, I think the doc reaches a good compromise: a short easy
> to understand english description that gives a clear idea of what all()
> is for, and a sample implementation for those who need/want the exact
> behaviour.
>
"any", "every", "no", etc. in the plain-English realm. The only way
around this is to have the definition focus on the *individual* element
of the iterable, not on the *collection* of elements. So I think Arnaud
is headed in the right direction. My own choice would be to ignore his
(very reasonable) objection to a definition that "makes it much less
obvious what the function is for", and go with these:
all(iterable) -- Return True if there does not exist an element in
iterable that is False
any(iterable) -- Return False if there does not exist an element in
iterable that is True
These definitions *imply* the functions' short-circuiting behavior, but
aren't as good as Arnaud's definition, which makes the short-circuiting
explicit.
-John
John O'Hagan
[...]
Agreed; having absorbed the answers to my original question, I now understand
why all('Robinson Crusoe' in books for books in []) - or with any object in
place of the string, or indeed just all([]) - doesn't return False, but,
vacuous truth notwithstanding (and it's apparently a not uncontroversial
notion), it doesn't seem fair to return True without so much as a
how's-your-father. ;)
An exception, or at least a specific mention of the case of empty iterables in
the docs (as Tim Chase suggested), would have baffled me less than my program
telling me that the same items were both ubiquitous and entirely absent from
a list of sets!
Of course, it's possible people out there depend on the present behaviour.
Regards,
John
Luis Alberto Zarrabeitia Gomez
Quoting John O'Hagan <rese...@johnohagan.com>:
> An exception, or at least a specific mention of the case of empty iterables
> in the docs (as Tim Chase suggested), would have baffled me less than my
> program telling me that the same items were both ubiquitous and entirely
> absent from a list of sets!
Well, they _were_ both ubiquitous and entirely absent from your list of sets:
both propositions are true. "Vacuous truth" doesn't mean that the answer is
incorrect, just that it is meaningless. The problem was not in the answer, but
in the question you asked.
> Of course, it's possible people out there depend on the present behaviour.
It's more than that. Python's following the rules here. Maybe it could be
documented better, for those without a background in logic/discrete mathematics,
but not changed.
Mark Dickinson
wrote:
> It's more than that. Python's following the rules here. Maybe it could be
> documented better, for those without a background in logic/discrete mathematics,
> but not changed.
Agreed.
I'd like to guess that in 93.7% of cases, when a programmer
has used all(seq) without having thought in advance about what the
right thing to do is when seq is empty, the current behaviour is
already the right one. I tried to test this hypothesis, but a
Google code search for uses of all() turned up very little
besides definitions. For example:
if all(t.already_filed() for t in my_tax_forms):
go_to_bed_happy()
else:
file_for_extension()
In the event that you didn't have any tax_forms, this
does the right thing.
The current definition also makes reasoning about programs and
program transformations easier, thanks to invariants like:
all(seq1 + seq2) === all(seq1) and all(seq2)
and
all(all(s) for s in seqs) === all(chain(*seqs))
and
any(not s for s in seq) == not all(seq)
These invariants wouldn't hold if all([]) were False, or raised
an exception.
IMO, the current behaviour meets both the practicality *and*
the purity criteria.
Mark
John O'Hagan
> On Apr 14, 7:21 pm, Luis Alberto Zarrabeitia Gomez <ky...@uh.cu>
>
> wrote:
> > It's more than that. Python's following the rules here. Maybe it could be
> > documented better, for those without a background in logic/discrete
> > mathematics, but not changed.
>
> Agreed.
>
> I'd like to guess that in 93.7% of cases, when a programmer
> has used all(seq) without having thought in advance about what the
> right thing to do is when seq is empty, the current behaviour is
> already the right one. I tried to test this hypothesis, but a
> Google code search for uses of all() turned up very little
> besides definitions. For example:
>
> if all(t.already_filed() for t in my_tax_forms):
> go_to_bed_happy()
> else:
> file_for_extension()
But what about this:
if all(evidence):
suspect_is_guilty
else:
suspect_is_innocent
:)
[...]
>
> The current definition also makes reasoning about programs and
> program transformations easier, thanks to invariants like:
>
> all(seq1 + seq2) === all(seq1) and all(seq2)
>
> and
>
> all(all(s) for s in seqs) === all(chain(*seqs))
>
> and
>
> any(not s for s in seq) == not all(seq)
>
> These invariants wouldn't hold if all([]) were False, or raised
> an exception.
>
[...]
OK, that's pretty compelling!
Regards,
John
Piet van Oostrum
>MD> On Apr 14, 7:21 pm, Luis Alberto Zarrabeitia Gomez <ky...@uh.cu>
>MD> wrote:
>>> It's more than that. Python's following the rules here. Maybe it could be
>>> documented better, for those without a background in logic/discrete mathematics,
>>> but not changed.
>MD> Agreed.
>MD> I'd like to guess that in 93.7% of cases, when a programmer
>MD> has used all(seq) without having thought in advance about what the
>MD> right thing to do is when seq is empty, the current behaviour is
>MD> already the right one. I tried to test this hypothesis, but a
>MD> Google code search for uses of all() turned up very little
>MD> besides definitions. For example:
>MD> if all(t.already_filed() for t in my_tax_forms):
>MD> go_to_bed_happy()
>MD> else:
>MD> file_for_extension()
>MD> In the event that you didn't have any tax_forms, this
>MD> does the right thing.
>MD> The current definition also makes reasoning about programs and
>MD> program transformations easier, thanks to invariants like:
>MD> all(seq1 + seq2) === all(seq1) and all(seq2)
>MD> and
>MD> all(all(s) for s in seqs) === all(chain(*seqs))
>MD> and
>MD> any(not s for s in seq) == not all(seq)
>MD> These invariants wouldn't hold if all([]) were False, or raised
>MD> an exception.
That is so because 'all' and 'any' are reduce (or fold) operations on the
logical operations 'and' and 'or', respectively. And the reduce
operation on an empty sequence should be the identity of the underlying
operation. The identity of 'and' is True and that of 'or' is False:
(x and True) == (True and x) == x
(x or False) == (False or x) == x
for every boolean x.
--
Piet van Oostrum <pi...@cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: pi...@vanoostrum.org
Steven D'Aprano
> Agreed; having absorbed the answers to my original question, I now
> understand why all('Robinson Crusoe' in books for books in []) - or with
> any object in place of the string, or indeed just all([]) - doesn't
> return False, but, vacuous truth notwithstanding (and it's apparently a
> not uncontroversial notion), it doesn't seem fair to return True without
> so much as a how's-your-father. ;)
I don't see why you call it unfair. It is very fair. You're just
unfortunate that *your* use-case was one of the times that the alternate
behaviour would be better, *and* you didn't know about vacuous truths. If
you had known, you wouldn't have been surprised -- but that's true about
everything. It is, in fact, a vacuous truth :)
http://en.wikipedia.org/wiki/Analytic-synthetic_distinction
I would be far more upset by all() raising an exception on an empty list!
And it is very simple to create a version of all() which doesn't return
true for vacuous truths.
def all2(sequence_or_iterator):
"""Like all() but returns False for vacuous truths."""
# Untested
it = iter(sequence_or_iterator)
try:
if not it.next(): return False
except StopIteration:
return False
return all(it)
> An exception, or at least a specific mention of the case of empty
> iterables in the docs (as Tim Chase suggested), would have baffled me
> less than my program telling me that the same items were both ubiquitous
> and entirely absent from a list of sets!
>
> Of course, it's possible people out there depend on the present
> behaviour.
You better believe it.
--
Steven
Steven D'Aprano
> In fact the doc is not just misleading, but plain wrong as the following
> shows:
>
>>>> def alwaystrue():
> ... while True: yield True
> ...
>>>> all(alwaystrue())
> [Python is stuck]
>
> The iterable alwaystrue() satisfies the property "all elements of the
> iterable are true", but all(alwaystrue()) does not return.
That's okay, the documentation doesn't say or imply that all() will
always return. Python doesn't make that promise about *any* function, and
it would be ridiculous to expect EVERY SINGLE Python function and class
to come with a warning that it won't return if you pass it an argument
that doesn't return.
def slow():
time.sleep(10**10)
return math.pi
math.cos(slow())
Is the documentation for cos() misleading because it doesn't warn that it
will take 300+ years to return? I don't think so.
Python guarantees that all() will return True if all the elements of the
iterable are true, but it doesn't make any promises about how much time
that will take. If there are six elements, and they're all true, you
can't know that they're all true until you have checked every one of
them. If there are six million elements, you have to check all six
million. If there are an infinite number of elements, how many do you
have to check, and how long will it take? This doesn't need to be spelled
out in the docs.
--
Steven
Aaron Brady
> On Tue, 14 Apr 2009, Mark Dickinson wrote:
> > On Apr 14, 7:21 pm, Luis Alberto Zarrabeitia Gomez <ky...@uh.cu>
>
> > wrote:
> > > It's more than that. Python's following the rules here. Maybe it could be
> > > documented better, for those without a background in logic/discrete
> > > mathematics, but not changed.
>
> > Agreed.
>
> > I'd like to guess that in 93.7% of cases, when a programmer
> > has used all(seq) without having thought in advance about what the
> > right thing to do is when seq is empty, the current behaviour is
> > already the right one. I tried to test this hypothesis, but a
> > Google code search for uses of all() turned up very little
> > besides definitions. For example:
>
> > if all(t.already_filed() for t in my_tax_forms):
> > go_to_bed_happy()
> > else:
> > file_for_extension()
>
> But what about this:
>
> if all(evidence):
> suspect_is_guilty
> else:
> suspect_is_innocent
>
> :)
> > program transformations easier, thanks to invariants like:
>
> > all(seq1 + seq2) === all(seq1) and all(seq2)
>
> > and
>
> > all(all(s) for s in seqs) === all(chain(*seqs))
>
> > and
>
> > any(not s for s in seq) == not all(seq)
>
> > These invariants wouldn't hold if all([]) were False, or raised
> > an exception.
>
> [...]
>
> OK, that's pretty compelling!
You could spell it out with a normal form like this:
all( x1, x2, ..., xn )=== if True and x1 and x2 and ... and xn.
any( ..., xn )=== if False or... or xn.
since 'if A' === 'if True and A'; and 'if A' === 'if False or A', then
the transformation should hold.
> if all(evidence):
> suspect_is_guilty
> else:
> suspect_is_innocent
>
> :)
Ha ha ha. However, if you try this: if sum( evidence )> 0.9* len
( evidence ), you will get an exception. Juries should recognize that
'verdict( evidence= set() )' is undefined, so either answer could be
evaluated to. Especially when you consider that testimony, and not
just court exhibits, are included in the evidence they consider. Or,
if you have, 'verdict( testimony= set([something]), evidence= set
() )', then the logic you presented isn't an accurate formulation of
the function you want. Fyi.
Dale Roberts
> ...
> I still prefer "Return False if any element of the iterable is
> not true" or "Return False if any element in the iterable is
> false" because that describes exactly what the algorithm does.
I agree that the original doc comment is not helpful as it stands
(even though the behavior of any() is of course correct!), and prefer
Tim's alternative. Since Python is used by programmers (hopefully!),
the doc comment should be directed toward that audience. It should be
unambiguous, and should not assume everyone has perfect knowledge of
mathematical logic operations, or that Python necessarily follows the
rules that a logician would expect (take the varying behavior of
"modulo" operators in various languages as an example).
Pure logic aside, if I was presented with the original comment
('Return True if all elements of the iterable are true.') as a
specification, as a programmer I would immediately have to ask what to
do in the case of an empty list. It might be that the user hadn't
thought about it, or would want to throw an exception, or return
False.
The doc should speak to the intended audience: programmers, who like
to make sure all bases and cases are covered.
dale
Raymond Hettinger
> to make sure all bases and cases are covered.
FWIW, I wrote the docs. The pure python forms were put in
as an integral part of the documentation. The first
sentence of prose was not meant to stand alone. It is a
lead-in to the code which makes explicit the short-circuiting
behavior and the behavior when the input is empty.
I will not change the sentence to "return false if any element
of the iterable is false." The negations make the sentence
hard to parse mentally and do not provide a motivation for
the use of the word "all". As it stands, the sentence provides
a clean lead-in for the code which is there "to make sure all
bases and cases are covered". I put code in the docs to make
the docs precise. We use code in the docs because it can
communicate clearly in cases where plain English prose is
either vague, awkward, hard to read, imprecise, or subject
to misinterpretation.
Of course, neither code nor accompanying prose help if prior
to reading them, a reader already has formed a strong but
incorrect mental picture of what the function does. If
someone has a priori convinced themselves that all([]) returns
False or has never thought about that case, then a cursory
reading of the docs is unlikely to disabuse them of that notion.
At any rate, all readers of this thread now seem to be
dialed-in as to why the current behavior is desirable.
I will probably leave the lead-in sentence as-is but may
add another sentence specifically covering the case for
an empty iterable.
Raymond
P.S. Now maybe we can start a new thread about why sum([])
returns zero ;-)
Tim Chase
> I will not change the sentence to "return false if any element
> of the iterable is false." The negations make the sentence
> hard to parse mentally
Just as a ribbing, that "return X if any element of the iterable
is X" is of the same form as the original. The negation is only
of the X, not of the sentence structure.
> I will probably leave the lead-in sentence as-is but may
> add another sentence specifically covering the case for
> an empty iterable.
as one of the instigators in this thread, I'm +1 on this solution.
Changing the implementation of all() would break waaaay too much
stuff, so I'm -1 on that. Adding a one-sentence clarification to
the docs is a heckuva lot easier.
> P.S. Now maybe we can start a new thread about why sum([])
> returns zero ;-)
NOOOOooooooo! (runs screaming) :-)
-tkc
Paul Rudin
> Changing the implementation of all() would break waaaay too much
> stuff...
Not to mention it clearly works correctly as is. *If* there is an issue
it is a documentation one... not an implementation one.
Dale Roberts
> Raymond Hettinger wrote:
> > I will not change the sentence to "return false if any element
> > of the iterable is false." The negations make the sentence
> > hard to parse mentally
>
> Just as a ribbing, that "return X if any element of the iterable
> is X" is of the same form as the original. The negation is only
> of the X, not of the sentence structure.
>
> > I will probably leave the lead-in sentence as-is but may
> > add another sentence specifically covering the case for
> > an empty iterable.
>
> as one of the instigators in this thread, I'm +1 on this solution.
Yes, I now appreciate the motivation for having the word "all" in the
text, and simply adding something like "or the iterable is empty"
might head off future confusion.
dale
John Posner
>> I will probably leave the lead-in sentence as-is but may
>> add another sentence specifically covering the case for
>> an empty iterable.
>
>
last licks on this one ...
Including the word "all" in the definition of "all()" is suboptimal.
Especially since the everyday meaning of "all" is ambiguous. Sure, leave
in the code-equivalent to clarify things, but why not clarify in text,
also? Here's a compromise:
all(iterable) -- Return True if all elements of the /iterable/ are
true -- more
precisely, if there does not exist an element of the iterable that is
False.
Equivalent to ...
Raymond Hettinger
You're welcome.
> As long as people are getting in their
> last licks on this one ...
>
> Including the word "all" in the definition of "all()" is suboptimal.
> Especially since the everyday meaning of "all" is ambiguous. Sure, leave
> in the code-equivalent to clarify things, but why not clarify in text,
> also? Here's a compromise:
No thanks. The word "all" is in the text for a reason. It provides
the motivation and mental context for the naming of the function.
The code is there to make the notion precise and it does its job fine.
I've already made a doc update to cover the case for an empty
iterable.
This is a simple function. Too much word-smithing actually makes it
harder to understand for normal users. Besides, nothing will truly
satisfy newsgroupies hell-bent on misreading the docs.
The more interesting challenges have been aimed at improving the
understandability of super(), str.split(), and itertools.groupby().
The toughest challenge so far has been to document descriptors in a
comprehensive and precise way:
http://users.rcn.com/python/download/Descriptor.htm
Raymond
Luis Zarrabeitia
/me wonders how no one on this thread suggested that before.
That seems like a pretty simple and clear fix to all this thread.
+100^2
--
Luis Zarrabeitia (aka Kyrie)
Fac. de Matemática y Computación, UH.
http://profesores.matcom.uh.cu/~kyrie
Roel Schroeven
> On Apr 16, 2:27 pm, Tim Chase <python.l...@tim.thechases.com> wrote:
>> Raymond Hettinger wrote:
>>> I will not change the sentence to "return false if any element
>>> of the iterable is false." The negations make the sentence
>>> hard to parse mentally
>> Just as a ribbing, that "return X if any element of the iterable
>> is X" is of the same form as the original. The negation is only
>> of the X, not of the sentence structure.
>>
>>> I will probably leave the lead-in sentence as-is but may
>>> add another sentence specifically covering the case for
>>> an empty iterable.
>> as one of the instigators in this thread, I'm +1 on this solution.
>
> [...] and simply adding something like "or the iterable is empty"
> might head off future confusion.
+1
--
The saddest aspect of life right now is that science gathers knowledge
faster than society gathers wisdom.
-- Isaac Asimov
Roel Schroeven
Antoon Pardon
> On Tue, 14 Apr 2009, Mark Dickinson wrote:
>> On Apr 14, 7:21 pm, Luis Alberto Zarrabeitia Gomez <ky...@uh.cu>
>>
>> wrote:
>> > It's more than that. Python's following the rules here. Maybe it could be
>> > documented better, for those without a background in logic/discrete
>> > mathematics, but not changed.
>>
>> Agreed.
>>
>> I'd like to guess that in 93.7% of cases, when a programmer
>> has used all(seq) without having thought in advance about what the
>> right thing to do is when seq is empty, the current behaviour is
>> already the right one. I tried to test this hypothesis, but a
>> Google code search for uses of all() turned up very little
>> besides definitions. For example:
>>
>> if all(t.already_filed() for t in my_tax_forms):
>> go_to_bed_happy()
>> else:
>> file_for_extension()
>
>
> But what about this:
>
> if all(evidence):
> suspect_is_guilty
> else:
> suspect_is_innocent
even if the evidence is not empty, the above wouldn't be
a good test, because you need enough evidence en enough
is not implied by all even if all is more than nothing.
--
Antoon.
Jani Hakala
> FWIW, I wrote the docs. The pure python forms were put in
> as an integral part of the documentation. The first
> sentence of prose was not meant to stand alone. It is a
> lead-in to the code which makes explicit the short-circuiting
> behavior and the behavior when the input is empty.
>
Someone else seems to have written "What's new in Python 2.5"
documention which states:
"Two new built-in functions, any() and all(), evaluate whether an
iterator contains any true or false values. any() returns True if any
value returned by the iterator is true; otherwise it will return
False. all() returns True only if all of the values returned by the
iterator evaluate as true."
This description of all() doesn't seem to be correct.
> I will probably leave the lead-in sentence as-is but may
> add another sentence specifically covering the case for
> an empty iterable.
>
Does this change cover the documentation returned by help(all) and
help(any)?
Jani Hakala
John O'Hagan
> On 2009-04-15, John O'Hagan <ma...@johnohagan.com> wrote:
> > On Tue, 14 Apr 2009, Mark Dickinson wrote:
[...]
> >> I'd like to guess that in 93.7% of cases, when a programmer
> >> has used all(seq) without having thought in advance about what the
> >> right thing to do is when seq is empty, the current behaviour is
> >> already the right one. I tried to test this hypothesis, but a
> >> Google code search for uses of all() turned up very little
> >> besides definitions. For example:
> >>
> >> if all(t.already_filed() for t in my_tax_forms):
> >> go_to_bed_happy()
> >> else:
> >> file_for_extension()
> >
> > But what about this:
> >
> > if all(evidence):
> > suspect_is_guilty
> > else:
> > suspect_is_innocent
>
> even if the evidence is not empty, the above wouldn't be
> a good test, because you need enough evidence en enough
> is not implied by all even if all is more than nothing.
One should probably file for an extension until one gets some new tax forms,
too, for that matter. It was just a facetious way of suggesting that it could
be seen as counter-intuitive to say that no evidence is all true (you trimmed
my smiley).
>>> evidence=()
>>> all(i is True for i in evidence)
True
>>> all(i is False for i in evidence)
True
>>> all(i is blue for i in evidence)
True
However don't worry, I've been thoroughly convinced that this behaviour is the
way it is (and should be) done, if for no other reason than that the
alternatives are even weirder!
John