[Python-ideas] Re: [Python-Dev] reversed enumerate
62 views
Skip to first unread message
Steven D'Aprano
Apr 1, 2020, 10:25:27 PM4/1/20
to pytho...@python.org, python...@python.org
Hi Ilya,
I'm not sure that this mailing list (Python-Dev) is the right place for
this discussion, I think that Python-Ideas (CCed) is the correct place.
For the benefit of Python-Ideas, I have left your entire post below, to
establish context.
[Ilya]
> I needed reversed(enumerate(x: list)) in my code, and have discovered
> that it wound't work. This is disappointing because operation is well
> defined.
It isn't really well-defined, since enumerate can operate on infinite
iterators, and you cannot reverse an infinite stream. Consider:
def values():
while True:
yield random.random()
a, b = reversed(enumerate(values())
What should the first pair of (a, b) be?
However, having said that, I think that your idea is not unreasonable.
`enumerate(it)` in the most general case isn't reversable, but if `it`
is reversable and sized, there's no reason why `enumerate(it)` shouldn't
be too.
My personal opinion is that this is a fairly obvious and straightforward
enhancement, one which (hopefully!) shouldn't require much, if any,
debate. I don't think we need a new class for this, I think enhancing
enumerate to be reversable if its underlying iterator is reversable
makes good sense.
But if you can show some concrete use-cases, especially one or two from
the standard library, that would help your case. Or some other languages
which offer this functionality as standard.
On the other hand, I think that there is a fairly lightweight work
around. Define a helper function:
def countdown(n):
while True:
yield n
n -= 1
then call it like this:
# reversed(enumerate(seq))
zip(countdown(len(seq)-1), reversed(seq)))
So it isn't terribly hard to work around this. But I agree that it would
be nice if enumerate encapsulated this for the caller.
One potentially serious question: what should `enumerate.__reversed__`
do when given a starting value?
reversed(enumerate('abc', 1))
Should that yield...?
# treat the start value as a start value
(1, 'c'), (0, 'b'), (-1, 'a')
# treat the start value as an end value
(3, 'c'), (2, 'b'), (1, 'a')
Something else?
My preference would be to treat the starting value as an ending value.
Steven
On Wed, Apr 01, 2020 at 08:45:34PM +0200, Ilya Kamenshchikov wrote:
> Hi,
>
> I needed reversed(enumerate(x: list)) in my code, and have discovered that
> it wound't work. This is disappointing because operation is well defined.
> It is also well defined for str type, range, and - in principle, but not
> yet in practice - on dictionary iterators - keys(), values(), items() as
> dictionaries are ordered now.
> It would also be well defined on any user type implementing __iter__,
> __len__, __reversed__ - think numpy arrays, some pandas dataframes, tensors.
>
> That's plenty of usecases, therefore I guess it would be quite useful to
> avoid hacky / inefficient solutions like described here:
> https://code.activestate.com/lists/python-list/706205/.
>
> If deemed useful, I would be interested in implementing this, maybe
> together with __reversed__ on dict keys, values, items.
>
> Best Regards,
> --
> Ilya Kamen
>
> -----------
> p.s.
>
> *Sketch* of what I am proposing:
>
> class reversible_enumerate:
>
> def __init__(self, iterable):
> self.iterable = iterable
> self.ctr = 0
>
> def __iter__(self):
> for e in self.iterable:
> yield self.ctr, e
> self.ctr += 1
>
> def __reversed__(self):
> try:
> ri = reversed(self.iterable)
> except Exception as e:
> raise Exception(
> "enumerate can only be reversed if iterable to
> enumerate can be reversed and has defined length."
> ) from e
>
> try:
> l = len(self.iterable)
> except Exception as e:
> raise Exception(
> "enumerate can only be reversed if iterable to
> enumerate can be reversed and has defined length."
> ) from e
>
> indexes = range(l-1, -1, -1)
> for i, e in zip(indexes, ri):
> yield i, e
>
> for i, c in reversed(reversible_enumerate("Hello World")):
> print(i, c)
>
> for i, c in reversed(reversible_enumerate([11, 22, 33])):
>
> print(i, c)
> _______________________________________________
> Python-Dev mailing list -- pytho...@python.org
> To unsubscribe send an email to python-d...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/NDDKDUDHE3J7SR7IPO3QXCVFSGE4BAV4/
> Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/LCZS4UUH7UEQN6NTW76UY23RCEITXQKB/
Code of Conduct: http://python.org/psf/codeofconduct/
I'm not sure that this mailing list (Python-Dev) is the right place for
this discussion, I think that Python-Ideas (CCed) is the correct place.
For the benefit of Python-Ideas, I have left your entire post below, to
establish context.
[Ilya]
> I needed reversed(enumerate(x: list)) in my code, and have discovered
> that it wound't work. This is disappointing because operation is well
> defined.
It isn't really well-defined, since enumerate can operate on infinite
iterators, and you cannot reverse an infinite stream. Consider:
def values():
while True:
yield random.random()
a, b = reversed(enumerate(values())
What should the first pair of (a, b) be?
However, having said that, I think that your idea is not unreasonable.
`enumerate(it)` in the most general case isn't reversable, but if `it`
is reversable and sized, there's no reason why `enumerate(it)` shouldn't
be too.
My personal opinion is that this is a fairly obvious and straightforward
enhancement, one which (hopefully!) shouldn't require much, if any,
debate. I don't think we need a new class for this, I think enhancing
enumerate to be reversable if its underlying iterator is reversable
makes good sense.
But if you can show some concrete use-cases, especially one or two from
the standard library, that would help your case. Or some other languages
which offer this functionality as standard.
On the other hand, I think that there is a fairly lightweight work
around. Define a helper function:
def countdown(n):
while True:
yield n
n -= 1
then call it like this:
# reversed(enumerate(seq))
zip(countdown(len(seq)-1), reversed(seq)))
So it isn't terribly hard to work around this. But I agree that it would
be nice if enumerate encapsulated this for the caller.
One potentially serious question: what should `enumerate.__reversed__`
do when given a starting value?
reversed(enumerate('abc', 1))
Should that yield...?
# treat the start value as a start value
(1, 'c'), (0, 'b'), (-1, 'a')
# treat the start value as an end value
(3, 'c'), (2, 'b'), (1, 'a')
Something else?
My preference would be to treat the starting value as an ending value.
Steven
On Wed, Apr 01, 2020 at 08:45:34PM +0200, Ilya Kamenshchikov wrote:
> Hi,
>
> I needed reversed(enumerate(x: list)) in my code, and have discovered that
> it wound't work. This is disappointing because operation is well defined.
> It is also well defined for str type, range, and - in principle, but not
> yet in practice - on dictionary iterators - keys(), values(), items() as
> dictionaries are ordered now.
> It would also be well defined on any user type implementing __iter__,
> __len__, __reversed__ - think numpy arrays, some pandas dataframes, tensors.
>
> That's plenty of usecases, therefore I guess it would be quite useful to
> avoid hacky / inefficient solutions like described here:
> https://code.activestate.com/lists/python-list/706205/.
>
> If deemed useful, I would be interested in implementing this, maybe
> together with __reversed__ on dict keys, values, items.
>
> Best Regards,
> --
> Ilya Kamen
>
> -----------
> p.s.
>
> *Sketch* of what I am proposing:
>
> class reversible_enumerate:
>
> def __init__(self, iterable):
> self.iterable = iterable
> self.ctr = 0
>
> def __iter__(self):
> for e in self.iterable:
> yield self.ctr, e
> self.ctr += 1
>
> def __reversed__(self):
> try:
> ri = reversed(self.iterable)
> except Exception as e:
> raise Exception(
> "enumerate can only be reversed if iterable to
> enumerate can be reversed and has defined length."
> ) from e
>
> try:
> l = len(self.iterable)
> except Exception as e:
> raise Exception(
> "enumerate can only be reversed if iterable to
> enumerate can be reversed and has defined length."
> ) from e
>
> indexes = range(l-1, -1, -1)
> for i, e in zip(indexes, ri):
> yield i, e
>
> for i, c in reversed(reversible_enumerate("Hello World")):
> print(i, c)
>
> for i, c in reversed(reversible_enumerate([11, 22, 33])):
>
> print(i, c)
> _______________________________________________
> Python-Dev mailing list -- pytho...@python.org
> To unsubscribe send an email to python-d...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/NDDKDUDHE3J7SR7IPO3QXCVFSGE4BAV4/
> Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/LCZS4UUH7UEQN6NTW76UY23RCEITXQKB/
Code of Conduct: http://python.org/psf/codeofconduct/
Andrew Barnert via Python-ideas
Apr 1, 2020, 11:45:13 PM4/1/20
to Steven D'Aprano, pytho...@python.org, python...@python.org
Before jumping in:
In many cases, when you want to reverse an enumerate, it’s small and fixed-sized, so there’s a trivial way to do this: Just store the enumerate iterator in a tuple, and tuples are reversible.
for idx, value in reversed(tuple(enumerate(stuff))):
But of course there are some cases where this isn’t appropriate, like enumerating a fixed-size but huge input.
> On Apr 1, 2020, at 19:23, Steven D'Aprano <st...@pearwood.info> wrote:
>
> [Ilya]
>> I needed reversed(enumerate(x: list)) in my code, and have discovered
>> that it wound't work. This is disappointing because operation is well
>> defined.
>
> It isn't really well-defined, since enumerate can operate on infinite
> iterators, and you cannot reverse an infinite stream.
...
> However, having said that, I think that your idea is not unreasonable.
> `enumerate(it)` in the most general case isn't reversable, but if `it`
> is reversable and sized, there's no reason why `enumerate(it)` shouldn't
> be too.
Agreed—but this is just a small piece of a much wider issue. Today, enumerate is always an Iterator. It’s never reversible. But it’s also not sized, or subscriptable, or in-testable, even if you give it inputs that are. And it’s not just enumerate—the same is true for map, filter, zip, itertools.islice, itertools.dropwhile, etc.
There’s no reason these things couldn’t all be views, just like the existing dict views (and other things like memoryview and third-party things like numpy array slices). In fact, they already are in Swift, and will be in C++20.
> My personal opinion is that this is a fairly obvious and straightforward
> enhancement, one which (hopefully!) shouldn't require much, if any,
> debate. I don't think we need a new class for this, I think enhancing
> enumerate to be reversable if its underlying iterator is reversable
> makes good sense.
Actually, that doesn’t work—it has to be Sized as well.
More generally, it’s rarely _quite_ as simple as just “views support the same operations as the things they view”. An enumerate can be a Sequence if its input is, but a filter can’t. A map with multiple inputs isn’t Reversible unless they’re all not just Reversible but Sized, although a map with only one input doesn’t need it to be Sized. And so on. But none of these things are hard, it’s just a bunch of work to go through all the input types for all the view types and write up the rules. (Or steal them from another language or library that already did that work…)
> But if you can show some concrete use-cases, especially one or two from
> the standard library, that would help your case. Or some other languages
> which offer this functionality as standard.
Agreed. I don’t think we need to wait until someone designs and writes a complete viewtools library and submits it for stdlib inclusion before we can consider adding just one extension to one iterator. But I do think we want to add the one(s) that are most useful if any, not just whichever ones people think of first. I’ve personally wanted to reverse a map or a filter more often than an enumerate, but examples would easily convince me that that’s just me, and reversing enumerate is more needed.
> One potentially serious question: what should `enumerate.__reversed__`
> do when given a starting value?
>
> reversed(enumerate('abc', 1))
I don’t think this is a problem. When you reversed(tuple(enumerate('abc', 1))) today, what do you get? You presumably don’t even need to look that up or try it out. It would be pretty confusing if it were different without the tuple.
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/CO6H6HLPNEN3Y53XJTS2QXQOZEZTABXM/
In many cases, when you want to reverse an enumerate, it’s small and fixed-sized, so there’s a trivial way to do this: Just store the enumerate iterator in a tuple, and tuples are reversible.
for idx, value in reversed(tuple(enumerate(stuff))):
But of course there are some cases where this isn’t appropriate, like enumerating a fixed-size but huge input.
> On Apr 1, 2020, at 19:23, Steven D'Aprano <st...@pearwood.info> wrote:
>
> [Ilya]
>> I needed reversed(enumerate(x: list)) in my code, and have discovered
>> that it wound't work. This is disappointing because operation is well
>> defined.
>
> It isn't really well-defined, since enumerate can operate on infinite
> iterators, and you cannot reverse an infinite stream.
> However, having said that, I think that your idea is not unreasonable.
> `enumerate(it)` in the most general case isn't reversable, but if `it`
> is reversable and sized, there's no reason why `enumerate(it)` shouldn't
> be too.
There’s no reason these things couldn’t all be views, just like the existing dict views (and other things like memoryview and third-party things like numpy array slices). In fact, they already are in Swift, and will be in C++20.
> My personal opinion is that this is a fairly obvious and straightforward
> enhancement, one which (hopefully!) shouldn't require much, if any,
> debate. I don't think we need a new class for this, I think enhancing
> enumerate to be reversable if its underlying iterator is reversable
> makes good sense.
More generally, it’s rarely _quite_ as simple as just “views support the same operations as the things they view”. An enumerate can be a Sequence if its input is, but a filter can’t. A map with multiple inputs isn’t Reversible unless they’re all not just Reversible but Sized, although a map with only one input doesn’t need it to be Sized. And so on. But none of these things are hard, it’s just a bunch of work to go through all the input types for all the view types and write up the rules. (Or steal them from another language or library that already did that work…)
> But if you can show some concrete use-cases, especially one or two from
> the standard library, that would help your case. Or some other languages
> which offer this functionality as standard.
> One potentially serious question: what should `enumerate.__reversed__`
> do when given a starting value?
>
> reversed(enumerate('abc', 1))
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Nuri Jung
Jan 21, 2021, 6:23:20 AM1/21/21
to python...@python.org
So, what is the conclusion? I also think reversed(enumerate(some_seq)) will be very useful in many cases.
It should:
1) work the same as reversed(tuple(enumerate(...))) for "reversible" objects as argument of enumerate,
2) raise TypeError if the object is not reversible.
Or, another option would be adding a "step" keyword argument to enumerate.
Then, reversing enumerate would be much easier, like this:
enumerate(reversed(some_seq), step=-1)
It should:
1) work the same as reversed(tuple(enumerate(...))) for "reversible" objects as argument of enumerate,
2) raise TypeError if the object is not reversible.
Or, another option would be adding a "step" keyword argument to enumerate.
Then, reversing enumerate would be much easier, like this:
enumerate(reversed(some_seq), step=-1)
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/BW2XKRZ4TSXCFULCS2EU33LLNMIXAL5T/
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Neil Girdhar
Jan 21, 2021, 1:19:20 PM1/21/21
to python-ideas
I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences. I think it's good idea, but it adds complexity and work, which I guess needs to be justified on a case-by-case basis.
In short, this has nothing to do with reversed. If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34], which could also be useful. I think it makes Python slightly more perfect and more beautiful.
Best,
Neil
Chris Angelico
Jan 21, 2021, 1:26:49 PM1/21/21
to python-ideas
On Fri, Jan 22, 2021 at 5:21 AM Neil Girdhar <miste...@gmail.com> wrote:
>
> I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences. I think it's good idea, but it adds complexity and work, which I guess needs to be justified on a case-by-case basis.
>
> In short, this has nothing to do with reversed. If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34], which could also be useful. I think it makes Python slightly more perfect and more beautiful.
>
list(enumerate(some_list)) will give you a sequence, if that's what you want.
>
> I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences. I think it's good idea, but it adds complexity and work, which I guess needs to be justified on a case-by-case basis.
>
> In short, this has nothing to do with reversed. If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34], which could also be useful. I think it makes Python slightly more perfect and more beautiful.
>
ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/WW3JHS234TJSXKUICY4PG2SPDTD3SBUP/
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Neil Girdhar
Jan 21, 2021, 1:42:39 PM1/21/21
to python-ideas, python-ideas
On Thu, Jan 21, 2021 at 1:26 PM Chris Angelico <ros...@gmail.com> wrote:
>
> On Fri, Jan 22, 2021 at 5:21 AM Neil Girdhar <miste...@gmail.com> wrote:
> >
> > I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences. I think it's good idea, but it adds complexity and work, which I guess needs to be justified on a case-by-case basis.
> >
> > In short, this has nothing to do with reversed. If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34], which could also be useful. I think it makes Python slightly more perfect and more beautiful.
> >
>
> list(enumerate(some_list)) will give you a sequence, if that's what you want.
Right. And reversed(list(enumerate(some_list)) also works. The point
>
> On Fri, Jan 22, 2021 at 5:21 AM Neil Girdhar <miste...@gmail.com> wrote:
> >
> > I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences. I think it's good idea, but it adds complexity and work, which I guess needs to be justified on a case-by-case basis.
> >
> > In short, this has nothing to do with reversed. If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34], which could also be useful. I think it makes Python slightly more perfect and more beautiful.
> >
>
> list(enumerate(some_list)) will give you a sequence, if that's what you want.
is that if you return sequence views (as Andrew mentioned), you don't
need to cast to list explicitly, and you don't pay the computational
cost of that either.
>
> ChrisA
> _______________________________________________
> Python-ideas mailing list -- python...@python.org
> To unsubscribe send an email to python-id...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at https://mail.python.org/archives/list/python...@python.org/message/WW3JHS234TJSXKUICY4PG2SPDTD3SBUP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> ---
> You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-ideas/3wGART2ECXQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to python-ideas...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/python-ideas/CAPTjJmpPchBQEWVP1Mt8UW3aaquTKx2kLVrxhc%3DV7LOpD%3Db7XQ%40mail.gmail.com.
Chris Angelico
Jan 21, 2021, 1:52:42 PM1/21/21
to python-ideas
On Fri, Jan 22, 2021 at 5:44 AM Neil Girdhar <miste...@gmail.com> wrote:
>
> On Thu, Jan 21, 2021 at 1:26 PM Chris Angelico <ros...@gmail.com> wrote:
> >
> > On Fri, Jan 22, 2021 at 5:21 AM Neil Girdhar <miste...@gmail.com> wrote:
> > >
> > > I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences. I think it's good idea, but it adds complexity and work, which I guess needs to be justified on a case-by-case basis.
> > >
> > > In short, this has nothing to do with reversed. If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34], which could also be useful. I think it makes Python slightly more perfect and more beautiful.
> > >
> >
> > list(enumerate(some_list)) will give you a sequence, if that's what you want.
>
> Right. And reversed(list(enumerate(some_list)) also works. The point
> is that if you return sequence views (as Andrew mentioned), you don't
> need to cast to list explicitly, and you don't pay the computational
> cost of that either.
>
With enumerate, you might be able to make a lazy sequence, but then
>
> On Thu, Jan 21, 2021 at 1:26 PM Chris Angelico <ros...@gmail.com> wrote:
> >
> > On Fri, Jan 22, 2021 at 5:21 AM Neil Girdhar <miste...@gmail.com> wrote:
> > >
> > > I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences. I think it's good idea, but it adds complexity and work, which I guess needs to be justified on a case-by-case basis.
> > >
> > > In short, this has nothing to do with reversed. If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34], which could also be useful. I think it makes Python slightly more perfect and more beautiful.
> > >
> >
> > list(enumerate(some_list)) will give you a sequence, if that's what you want.
>
> Right. And reversed(list(enumerate(some_list)) also works. The point
> is that if you return sequence views (as Andrew mentioned), you don't
> need to cast to list explicitly, and you don't pay the computational
> cost of that either.
>
people will ask for map to be able to return a sequence too - and
that's going to definitely run into problems. It's much safer to be
explicit about it: if you want a list, ask for a list.
In the rare instances where you actually need an enumerated lazy list
(ie where the cost of an eager list is too high AND it needs to be
enumerated), it's not that hard to whip up a thing that gives back the
index as well:
class Enumerated:
def __init__(self, basis): self.basis = basis
def __getitem__(self, idx): return (idx, self.basis[idx])
def __len__(self): return len(self.basis)
That ought to do most or all of what you want. It's iterable and
reversible, and you can index it directly.
ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Christopher Barker
Jan 21, 2021, 3:15:29 PM1/21/21
to Neil Girdhar, python-ideas, python-ideas
On Thu, Jan 21, 2021 at 10:43 AM Neil Girdhar <miste...@gmail.com> wrote:
> > I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences.
I'm not so sure -- I don't think I want a sequence returned. In fact, in this case, just the opposite. If you wanted a sequence, you could simply do:
reversed(list(enumerate(the_iterable)))
Rather, you want (or I want, anyway) is for various iterators to be able to do things that require a Sequence, when given a Sequence. Or, maybe not even a full Sequence, but rather, some particular aspect of a sequence -- like the length, or indexability.
enumerate() with a reversed flag is one. Another, I'd like is islice to support negative indexing for indexing from the end. There are probably many others.
I'd love to see these supported. For this example:
enumerate(an_iterable, reversed=True)
would "just work" if an_iterable had a __len__ and a __getitem__.
It would raise a TypeError if passed a non-lengthed, or non-indexable iterable.
-CHB
> > In short, this has nothing to do with reversed.
I agree.
>> If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34],
As Chris A mentioned, it's quite easy to wrap list() or tuple() around it if you want that.
- Chris B
--
Christopher Barker, PhD (Chris)
Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
Neil Girdhar
Jan 21, 2021, 4:29:38 PM1/21/21
to Christopher Barker, python-ideas, python-ideas
On Thu, Jan 21, 2021 at 3:13 PM Christopher Barker <pyth...@gmail.com> wrote:
>
> On Thu, Jan 21, 2021 at 10:43 AM Neil Girdhar <miste...@gmail.com> wrote:
>>
>> > > I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences.
>
>
> I'm not so sure -- I don't think I want a sequence returned. In fact, in this case, just the opposite. If you wanted a sequence, you could simply do:
>
> reversed(list(enumerate(the_iterable)))
>
> Rather, you want (or I want, anyway) is for various iterators to be able to do things that require a Sequence, when given a Sequence. Or, maybe not even a full Sequence, but rather, some particular aspect of a sequence -- like the length, or indexability.
>
> enumerate() with a reversed flag is one. Another, I'd like is islice to support negative indexing for indexing from the end. There are probably many others.
>
> I'd love to see these supported. For this example:
>
> enumerate(an_iterable, reversed=True)
Don't you think this is over-complicated?
>
> On Thu, Jan 21, 2021 at 10:43 AM Neil Girdhar <miste...@gmail.com> wrote:
>>
>> > > I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences.
>
>
> I'm not so sure -- I don't think I want a sequence returned. In fact, in this case, just the opposite. If you wanted a sequence, you could simply do:
>
> reversed(list(enumerate(the_iterable)))
>
> Rather, you want (or I want, anyway) is for various iterators to be able to do things that require a Sequence, when given a Sequence. Or, maybe not even a full Sequence, but rather, some particular aspect of a sequence -- like the length, or indexability.
>
> enumerate() with a reversed flag is one. Another, I'd like is islice to support negative indexing for indexing from the end. There are probably many others.
>
> I'd love to see these supported. For this example:
>
> enumerate(an_iterable, reversed=True)
It's possible for reversed(enumerate(...)) to just work if enumerate
of a sequence were to return a sequence view. Then you would also get
all the other sequence operations for free like enumerate(...)[23:27],
len(enumerate(...)), etc.
>
> would "just work" if an_iterable had a __len__ and a __getitem__.
>
> It would raise a TypeError if passed a non-lengthed, or non-indexable iterable.
>
> -CHB
>
> > > In short, this has nothing to do with reversed.
>
> I agree.
>
> >> If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34],
>
> As Chris A mentioned, it's quite easy to wrap list() or tuple() around it if you want that.
cost. However, as was suggested by Andrew in this thread, but has
come up many times before, the native iteration tools could return
views. This saves the user having to build her own custom classes
(along with tests, etc.). I happen to think this is simpler and more
beautiful.
Best,
Neil
>
Chris Angelico
Jan 21, 2021, 6:49:10 PM1/21/21
to python-ideas
On Fri, Jan 22, 2021 at 8:31 AM Neil Girdhar <miste...@gmail.com> wrote:
> It's possible for reversed(enumerate(...)) to just work if enumerate
> of a sequence were to return a sequence view. Then you would also get
> all the other sequence operations for free like enumerate(...)[23:27],
> len(enumerate(...)), etc.
That would break backward compatibility. The behaviour of enumerate()
> It's possible for reversed(enumerate(...)) to just work if enumerate
> of a sequence were to return a sequence view. Then you would also get
> all the other sequence operations for free like enumerate(...)[23:27],
> len(enumerate(...)), etc.
at the moment isn't compatible with it just suddenly being a sequence.
It would have to have some way to be told "please return a sequence".
We currently have a way to say "please return an eager sequence", so
the only thing we don't easily have is "please return a lazy
sequence", and that's possible with about four lines of custom class.
Note that slicing is NOT easy. The proposed semantics for a reversed
enumeration would make slicing extremely odd.
> > As Chris A mentioned, it's quite easy to wrap list() or tuple() around it if you want that.
>
> Yes, you can alternatively wrap with list and pay the computational
> cost. However, as was suggested by Andrew in this thread, but has
> come up many times before, the native iteration tools could return
> views. This saves the user having to build her own custom classes
> (along with tests, etc.). I happen to think this is simpler and more
> beautiful.
view, but let's look at map. Should that return a view? If so, does
map(...)[4] cache the result or not? If it does, you're paying the
memory cost of (potentially) the full eager list, in which case you
may as well just make the list. If it doesn't, you break expectations
badly by having it call the function more than once, potentially
giving different results. Either way, it's far better to write your
own, giving you the freedom to define the semantics the way you want
to.
It's even worse with filter(). It can't possibly know which is the 5th
element without counting from the start ("hmm, that's Fire... Earth...
Water... Air... Boron. Ah! Boron!").
What about itertools.tee()? I think the whole point of that is to
hybridize laziness and caching, but if you're going to give it a
sequence, you may as well just get multiple iterators.
I'm eyeballing itertools and really not seeing any that would benefit
from magically returning sequences. Several of them specifically exist
to provide sequence-like behaviour on non-sequence iterables (eg
islice), and others don't have a direct correlation between input
index and output index (eg dropwhile) or have other discrepancies (eg
takewhile).
Rather than try to propose that ALL native iteration tools return
views, find one that would actually benefit, and propose that.
Unfortunately, I think you'll run into the same problem that it's no
longer backward compatible, so it's still going to need a spelling to
request that it behave in sequence view mode, which is probably best
written as something like "import seqview; seqview.map(...)".
ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Random832
Jan 21, 2021, 7:09:27 PM1/21/21
to Python-ideas Python-ideas
On Thu, Jan 21, 2021, at 18:48, Chris Angelico wrote:
> Note that slicing is NOT easy. The proposed semantics for a reversed
> enumeration would make slicing extremely odd.
What proposed semantics? You were the one who posted a pure-python implementation that didn't bother to implement slicing.
> Note that slicing is NOT easy. The proposed semantics for a reversed
> enumeration would make slicing extremely odd.
It's easy enough to add slicing to your Enumerated class concept, though.
class Enumerated:
def __init__(self, basis, indices=None):
self.basis = basis
self.indices = indices if indices is not None else range(len(basis))
def __getitem__(self, idx):
if isinstance(idx, slice):
return Enumerated(self.basis, indices=self.indices[idx])
else:
return (self.indices[idx], self.basis[self.indices[idx]])
def __len__(self):
return len(self.indices)
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/ZV3DG5HEM34WY2C5ETO744XBP4SAQENU/
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Chris Angelico
Jan 21, 2021, 7:19:38 PM1/21/21
to Python-ideas Python-ideas
On Fri, Jan 22, 2021 at 11:10 AM Random832 <rand...@fastmail.com> wrote:
>
> On Thu, Jan 21, 2021, at 18:48, Chris Angelico wrote:
> > Note that slicing is NOT easy. The proposed semantics for a reversed
> > enumeration would make slicing extremely odd.
>
> What proposed semantics? You were the one who posted a pure-python implementation that didn't bother to implement slicing.
Ah, my bad. This thread started out on python-dev before migrating to
>
> On Thu, Jan 21, 2021, at 18:48, Chris Angelico wrote:
> > Note that slicing is NOT easy. The proposed semantics for a reversed
> > enumeration would make slicing extremely odd.
>
> What proposed semantics? You were the one who posted a pure-python implementation that didn't bother to implement slicing.
-ideas, and I believe the proposed semantics may have been on one of
the early -dev posts.
The proposal was, effectively, to be the same as enumerating, making a
concrete list, and then reversing (only, without actually building the
list, of course). So enumerate("abc") yields [(0,'a'),(1,'b'),
(2,'c')], and iterating backwards should yield those same tuples in
reverse order.
But if you take the reversed enumeration, iterate over it a bit, and
then slice it, the semantics will be extremely bizarre.
> It's easy enough to add slicing to your Enumerated class concept, though.
>
> class Enumerated:
> def __init__(self, basis, indices=None):
> self.basis = basis
> self.indices = indices if indices is not None else range(len(basis))
> def __getitem__(self, idx):
> if isinstance(idx, slice):
> return Enumerated(self.basis, indices=self.indices[idx])
> else:
> return (self.indices[idx], self.basis[self.indices[idx]])
> def __len__(self):
> return len(self.indices)
especially with some iteration in between.
The precise semantics for mixing iteration and slicing are complex
enough and variable enough that you may as well just go for a concrete
list at that point.
ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/VTSZZK3RKSKCEG7CFHYT4TZXLMP37P3I/
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Neil Girdhar
Jan 21, 2021, 7:46:08 PM1/21/21
to python-ideas, Python-ideas Python-ideas
that we're not talking about the same thing at all.
If enumerate(some_sequence) returns a sequence view, iterating over
that sequence view does not advance it—just like how DictViews are not
altered by iteration. Same thing if reversed(some_sequence) returns a
sequence view.
So, reversed(enumerate("abc"))[2] would just be (0, 'a').
You're right that there are some minor incongruities. The only one I
can think of off-hand is bool(enumerate(some_sequence)) would be false
when the sequence is empty.
As for candidates for sequence views, I'd like to see reversed and
enumerate to start since I can remember having to cast to list many
times and it would be convenient not to have to. From itertools, it
would be useful for thecombinatorics operators (combinations,
permutations, product, etc.) to be sequence views. In fact, there are
convenience functions for looking them up by index in more-itertools
as this comes up a lot (at least it did for me back when I did
contests).
>
> ChrisA
> _______________________________________________
> Python-ideas mailing list -- python...@python.org
> To unsubscribe send an email to python-id...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at https://mail.python.org/archives/list/python...@python.org/message/VTSZZK3RKSKCEG7CFHYT4TZXLMP37P3I/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-ideas/3wGART2ECXQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to python-ideas...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/python-ideas/CAPTjJmpboVPni3FimaTh%3DNo_GBUfTmozE4kQhXBPQCrsgmkSJg%40mail.gmail.com.
>
> ---
> You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-ideas/3wGART2ECXQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to python-ideas...@googlegroups.com.
Chris Angelico
Jan 21, 2021, 8:37:36 PM1/21/21
to Python-ideas Python-ideas
On Fri, Jan 22, 2021 at 11:48 AM Neil Girdhar <miste...@gmail.com> wrote:
> If enumerate(some_sequence) returns a sequence view, iterating over
> that sequence view does not advance it—just like how DictViews are not
> altered by iteration. Same thing if reversed(some_sequence) returns a
> sequence view.
Then that's backward incompatible with current behaviour, and cannot
> If enumerate(some_sequence) returns a sequence view, iterating over
> that sequence view does not advance it—just like how DictViews are not
> altered by iteration. Same thing if reversed(some_sequence) returns a
> sequence view.
be done just as "it's a sequence so return a sequence view". Hence the
suggestion of a parameter to control this - or, my preferred way: a
"seqview" module from which you can import a sequence-view-only
version of various iterator functions.
ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Random832
Jan 22, 2021, 12:04:36 AM1/22/21
to Python-ideas Python-ideas
On Thu, Jan 21, 2021, at 19:19, Chris Angelico wrote:
> Yeah, I don't think that'll work if you slice more than once,
> especially with some iteration in between.
I think part of the point of this implementation [or the other person's suggestion of having a "collection view", which amounts to the same thing] is that iterating wouldn't consume items anymore. Which does make it incompatible with enumerate currently returning an iterator, though, in situations where the iteration is dropped and picked up again.
> Yeah, I don't think that'll work if you slice more than once,
> especially with some iteration in between.
> The precise semantics for mixing iteration and slicing are complex
> enough and variable enough that you may as well just go for a concrete
> list at that point.
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/D67HZZ2QX22SZ427Y6UKAB44UHIKHTL3/
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Reply all
Reply to author
Forward
0 new messages