Allow __len__ to return infinity

[Ram Rachum]

I'd like to have some objects that return infinity from their __len__ method. Unfortunately the __len__ method may only return an int, and it's impossible to represent an infinity as an int. Do you think that Python could allow returning infinity from __len__?

Thanks,

Ram.

[Steven D'Aprano]

"Could"? Of course. Almost anything is possible.

"Should"? No. Allowing __len__ to return float('inf') would imply one of
two alternatives, both equally unpalatable.

(1) __len__ can return *any* float, regardless of value, including
lengths of 0.5, NAN, 1e300, etc. This is undesirable because lengths of
sequences should be positive or zero whole numbers, not arbitrary
floating point values like 4.5.

(2) __len__ cannot return any float, but only INF. Which means that the
condition that len() only returns ints will be broken in the most
surprising way, with a single exception.


I can see a few alternatives:


- Perhaps ints should grow a pair of special values, +INF and -INF. I'm
willing to be persuaded that this is a good idea.

- Or perhaps __len__ could loosen the restriction that the value
returned is non-negative. Perhaps -1 (or any negative length) could
stand in for "infinitely long". Although I think that would be
error-prone and cause more trouble than it solves.

- If not, perhaps you could use sys.maxsize as a stand-in for
"infinitely long". After all, if a sequence has 2147483647 items, that's
effectively infinite for most purposes.


There's another reason: the current implementation of len() in CPython
requires that the length fit in a C long:

py> class X:
... def __len__(self):
... return sys.maxsize + 1
...
py> x = X()
py> len(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'int' into an index-sized integer


I'm curious what your use-case for len() returning INF might be.


--
Steven
_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Ram Rachum]

I think that allowing int('inf') might be the best solution.

My use case is a `LazyTuple`, which is a smart object that wraps an iterator. These can sometimes have infinite lengths. (In case of an infinite iterator.)

--

---
You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-ideas/nFYbEpHrjlk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to python-ideas...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Paul Moore]

On 25 February 2014 11:18, Steven D'Aprano <st...@pearwood.info> wrote:
> (1) __len__ can return *any* float, regardless of value, including
> lengths of 0.5, NAN, 1e300, etc. This is undesirable because lengths of
> sequences should be positive or zero whole numbers, not arbitrary
> floating point values like 4.5.
>
> (2) __len__ cannot return any float, but only INF. Which means that the
> condition that len() only returns ints will be broken in the most
> surprising way, with a single exception.

Either of these would break range(len(x)) at a minimum, and probably
most other code that uses len(). So you'd be left with code having to
identify "objects that only return finite len" and "objects that could
have one of the new len values". And if the OP can detect that, he can
just do so in his code and special case his objects, rather than
changing the len protocol.

Paul

[Antoine Pitrou]

The question is what that would achieve? Consumers of len() expect it
to return a finite integer, so returning infinite would break lots of
existing code.

If you're willing to be practical, you can still return sys.maxsize,
for example (which is not far from infinite on 64-bit systems ;-)).

Regards

Antoine.

[Ram Rachum]

This actually makes me think that `range(int('inf'))` is a more elegant construct than `itertools.count()`. Also `range(x, int('inf'))` for `itertools.count(x)`, and then you have `range(x, int('inf'), y)` or `range(0, int('-inf'), -1)` which `itertools.count` can't cover.

[Peter Otten]

Ram Rachum wrote:

> This actually makes me think that `range(int('inf'))` is a more elegant
> construct than `itertools.count()`. Also `range(x, int('inf'))`
> for `itertools.count(x)`,

You could achieve that with range(None) or range(start, None)
which would be similar to slices like items[start:None] aka items[start:].

> and then you have `range(x, int('inf'), y)` or `range(0, int('-inf'), -1)`
> which `itertools.count` can't cover.

This *is* covered by count():

>>> [x for x in itertools.islice(itertools.count(step=-2), 10)]
[0, -2, -4, -6, -8, -10, -12, -14, -16, -18]

All you save is one import. In return you'll see your applications break in
new and interesting ways ;)

[Ram Rachum]

I stand corrected.

[Antony Lee]

This is similar to another idea I had some time ago: add keyword-argument support for range, and let range(start=x, [step=y]) (stop being equal to None) map to itertools.count(start=x, [step=y]).  No need for inf the way.

[Greg Ewing]

Steven D'Aprano wrote:
> (1) __len__ can return *any* float, regardless of value, including
> lengths of 0.5, NAN, 1e300, etc. This is undesirable because lengths of
> sequences should be positive or zero whole numbers,

What about things other than sequences?

For a vector type, for example, it would make sense
for len(v) to return the magnitude of the vector.

--
Greg

[Steven D'Aprano]

On Wed, Feb 26, 2014 at 12:40:59PM +1300, Greg Ewing wrote:
> Steven D'Aprano wrote:
> >(1) __len__ can return *any* float, regardless of value, including
> >lengths of 0.5, NAN, 1e300, etc. This is undesirable because lengths of
> >sequences should be positive or zero whole numbers,
>
> What about things other than sequences?
>
> For a vector type, for example, it would make sense
> for len(v) to return the magnitude of the vector.

I don't think it does. Despite the similarities in names, I don't think
Python the language should conflate *length of a sequence or mapping*
with *arbitrary measures of length*. len(x) by historical precedent
returns the length of a sequence/mapping/set, where the result returned
is a non-negative integer value. If you want some other definition of
length, such as the length of a vowel or the length of a polynomial, or
some other metric such as Manhattan distance or Chebyshev distance, or
even vector magnitude, you can create your own function or method and
call it (say) length().



--
Steven

[Ethan Furman]

On 02/25/2014 03:40 PM, Greg Ewing wrote:
> Steven D'Aprano wrote:
>> (1) __len__ can return *any* float, regardless of value, including lengths of 0.5, NAN, 1e300, etc. This is
>> undesirable because lengths of sequences should be positive or zero whole numbers,
>
> What about things other than sequences?
>
> For a vector type, for example, it would make sense
> for len(v) to return the magnitude of the vector.

Very good point. To which I reply: singledispatch ! ;)

http://docs.python.org/dev/whatsnew/3.4.html#whatsnew-singledispatch

--
~Ethan~

[Andrew Barnert]

I think my original reply got swallowed by Google Groups again, so apologies if this is a repeat to some or all…

From: Ram Rachum <ram.r...@gmail.com>
Sent: Tuesday, February 25, 2014 2:02 AM

>I'd like to have some objects that return infinity from their __len__ method. Unfortunately the __len__ method may only return an int, and it's impossible to represent an infinity as an int. Do you think that Python could allow returning infinity from __len__?

What code do you have that needs to distinguish between actually infinite iterables and potentially infinite ones?

Presumably you still want iterators to raise a TypeError rather than try to guess, so as soon as you pass an infinite iterable through a genexpr or itertools function it's going to turn into a length-less iterable.

Are you looking for this as a sort of intermediate step from length-less iterables to the kind of lazy lists that some functional languages have? You actually can build a lazy list (implementing Sequence by storing a list of already-seen-prefix and an iterator of not-yet-seen values) pretty easily (and write a lazytools module full of functions like the relevant itertools functions). I did this a few years back, but never found a good use for it and abandoned it. 

Anyway, if so, a lazy list that doesn't know its length already needs to say so by raising an exception from __len__, and a lazy list that knows its length is infinite can do the same thing, and I couldn't think of any uses cases where that would be a problem, even though I expected there would be some. So, if you have one, I'd love to hear it.