RELEASED: Python 2.3a2
Guido van Rossum
2.3. Much improved since the first alpha, chockfull of things you'd
like to check out:
Some highlights of what's new since 2.3a1:
- fixed mishandling of negated hex/oct literals (PEP 237)
- a new, more efficient pickling protocol (PEP 307)
- functions have a __module__ attribute
- new module itertools: efficient looping constructs
- many improvements to the new datetime module
New since Python 2.2:
- Many new and improved library modules, e.g. sets, heapq, datetime,
textwrap, optparse, logging, bsddb, bz2, tarfile,
ossaudiodev, and a new random number generator based on the highly
acclaimed Mersenne Twister algorithm (with a period of 2**19937-1!).
- New builtin enumerate(): an iterator yielding (index, item) pairs.
- Extended slices, e.g. "hello"[::-1] returns "olleh".
- Universal newlines mode for reading files (converts \r, \n and \r\n
all into \n).
- Source code encoding declarations. (PEP 263)
- Import from zip files. (PEP 273 and PEP 302)
- FutureWarning issued for "unsigned" operations on ints. (PEP 237)
- Faster list.sort() is now stable.
- Unicode filenames on Windows.
- Karatsuba long multiplication (running time O(N**1.58) instead of
O(N**2)).
We request widespread testing of this release but don't recommend
using it for production situations. Alpha releases contain bugs; in
addition, new APIs are not yet considered stable and may change until
the first beta release. If you have an important Python application,
we strongly recommend that you try it out with an alpha release and
report any incompatibilities or other problems you may encounter, so
that they can be fixed before the final release. To report problems,
use the SourceForge bug tracker:
http://sourceforge.net/tracker/?group_id=5470&atid=105470
Enjoy!
--Guido van Rossum (home page: http://www.python.org/~guido/)
Rene Pijlman
>- Extended slices, e.g. "hello"[::-1] returns "olleh".
This is a troll, right? :-)
--
René Pijlman
Wat wil jij leren? http://www.leren.nl
Alexander Schmolck
>
> - new module itertools: efficient looping constructs
>
I've got one general comment on the new itertools (which I think really is a
great addition to the standard library) and a few suggestions for additions.
Wouldn't `xfilter`, `xmap` etc. be a better naming convention than
`ifilter`, `imap` etc? I'd propose this change for 2 reasons:
1) the 'x' prefix is already used for `lazy` constructs (such as xrange and
xreadlines) which are in pretty much every way equivalent to generators.
2) `i` is currently used to signify `in-place` i.e. a destructive operation,
(e.g. __iadd__ etc.). It is often useful to have both a destructive and a
nondestructive variant of a function, sort e.g. would be a prime example
and I can't think of another reasonable convention in python than to prefix
i (for in-place).
Admittedly, the most "pythonic" approach is to have just a single,
destructive function that returns `None`, but this is often prohibitively
incovinient, so that uncountable destructive `sort` functions for lists
have been written, leading to ambiguity. A convention to distinguishes
destructive and nondestructive functions at least avoids that, e.g.:
def isort(l, cmpfunc=None):
if cmpfunc:
l.sort(cmpfunc)
else:
l.sort()
return l
def sort(seq, cmpfunc=None):
return isort(list(seq), cmpfunc=None)
Finally, among the tools I've cooked up myself over time and that aren't
already covered by what's in the new module, I find the following 2
(especially the first) to be the most useful and worthy of addition:
def xgroup(iter,n=2):
r"""Iterate `n`-wise (default pairwise) over `iter`.
Examples:
>>> list(xgroup(range(9), 3))
[(0, 1, 2), (3, 4, 5), (6, 7, 8)]
"""
last = []
for elt in iter:
last.append(elt)
if len(last) == n: yield tuple(last); last = []
def xwindow(iter, n=2, s=1):
r"""Move an `n`-item (default 2) windows `s` steps (default 1) at a time
over `iter`.
Examples:
>>> list(xwindow(range(6),2))
[(0, 1), (1, 2), (2, 3), (3, 4), (4, 5)]
"""
assert n >= s
last = []
for elt in iter:
last.append(elt)
if len(last) == n: yield tuple(last); last=last[s:]
A last thing, I think a `cycle` generator would be an really useful idiom to
have, as it provides a much more readable and less errorprone way to achieve
many of the things one normally has to use an awkward modulus construct
for. If I understand the rationale for not providing one correctly, then it is
because the `straightforward` (and non-storage allocating) implementation:
def xcycle(seq):
while 1:
for item in seq: yield item
doesn't work as one might expect if `seq` is e.g. an iterator (which BTW sort
of seems to contradict the other reason for not providing it, namely that "the
tool is readily constructible using pure Python" -- it is easy to overlook
this subtleness). Wouldn't the following work, mostly without surprising
overheads?
def xcycle(seq):
if not operator.isSequenceType(seq):
seq = tuple(seq) # or [x for x in seq]
while 1:
for item in seq: yield item
Since I'd guess that the most common usage scenario would be something like
``cycle([1,2,3])`` (for which there is no suprising storage allocation) I
don't think that that many expectations would be violated, after all.
alex
Jp Calderone
> Guido van Rossum:
> >- Extended slices, e.g. "hello"[::-1] returns "olleh".
>
> This is a troll, right? :-)
>
What makes you say that?
Jp
--
up 11 days, 22:28, 2 users, load average: 0.04, 0.03, 0.00
Inyeol Lee
> On Thu, Feb 20, 2003 at 10:58:17AM +0100, Rene Pijlman wrote:
> > >- Extended slices, e.g. "hello"[::-1] returns "olleh".
> >
> > This is a troll, right? :-)
> >
>
[::-1] ?
Inyeol...
A.M. Kuchling
Inyeol Lee <inyeo...@siimage.com> wrote:
> [::-1] ?
sequence[start:stop:stride] has been legal Python syntax since 1.4beta2, but
only Numeric Python ever supported the stride argument. 2.3 changes
Python's built-in sequence types to support strides; they never did before.
Hey, at least no one will ask for a reverse method on strings any more...
--amk (www.amk.ca)
LaTeX2HTML is pain.
-- Fred Drake in a documentation checkin message, 14 Mar 2000
Jim Meyer
> On Thu, Feb 20, 2003 at 01:10:42PM -0500, Jp Calderone wrote:
> > On Thu, Feb 20, 2003 at 10:58:17AM +0100, Rene Pijlman wrote:
> > > >- Extended slices, e.g. "hello"[::-1] returns "olleh".
> > >
> > > This is a troll, right? :-)
> > >
> >
>
> [::-1] ?
Looks like a four-eyed smirking smiley to me. ;]
--j
--
Jim Meyer, Geek at Large jme...@pdi.com
Tim Peters
> sequence[start:stop:stride] has been legal Python syntax since
> 1.4beta2, but only Numeric Python ever supported the stride argument.
> did before.
> Hey, at least no one will ask for a reverse method on strings any more...
Heh. The correspondents in the "str.reverse" Feature Request weren't
persuaded that slicing was adequate (despite agreeing that it worked to
reverse a string):
http://sf.net/tracker/?group_id=5470&atid=355470&func=detail&aid=494240
Chad Netzer
> Guido van Rossum <gu...@python.org> writes:
> > - new module itertools: efficient looping constructs
> Wouldn't `xfilter`, `xmap` etc. be a better naming convention than
> `ifilter`, `imap` etc? I'd propose this change for 2 reasons:
FWIW, I agree with your reasoning here. Since we are stuck with xrange,
the new names should be consistent.
> Finally, among the tools I've cooked up myself over time and that aren't
> already covered by what's in the new module, I find the following 2
> (especially the first) to be the most useful and worthy of addition:
>
> def xgroup(iter,n=2):
[snip]
> def xwindow(iter, n=2, s=1):
[snip]
> A last thing, I think a `cycle` generator would be an really useful idiom to
> have, as it provides a much more readable and less errorprone way to achieve
> many of the things one normally has to use an awkward modulus construct
> for.
I like xgroup and xcycle, but xwindow could be a confusing name (and is
arguably less often needed than xgroup). Any alternative names for
xwindow? Also, is there any (non-obfuscated) way to get xgroup()
functionality from zip()? It doesn't seem so.
At least there is now an izip() in itertools (although, again, I would
prefer the 'x' naming, or the addition of irange() and the deprecation
of xrange())
--
Bay Area Python Interest Group - http://www.baypiggies.net/
Chad Netzer
(any opinion expressed is my own and not NASA's or my employer's)
Inyeol Lee
> On Thu, 20 Feb 2003 10:40:17 -0800,
> Inyeol Lee <inyeo...@siimage.com> wrote:
> > [::-1] ?
>
> sequence[start:stop:stride] has been legal Python syntax since 1.4beta2, but
> only Numeric Python ever supported the stride argument. 2.3 changes
> Python's built-in sequence types to support strides; they never did before.
> Hey, at least no one will ask for a reverse method on strings any more...
I was just talking about troll(?)-like smiley, [::-1] with Jp.
Anyway, thanks.
Inyeol...
Raymond Hettinger
> I've got one general comment on the new itertools (which I think really is
a
> great addition to the standard library)
Thanks.
> Wouldn't `xfilter`, `xmap` etc. be a better naming convention than
> `ifilter`, `imap` etc?
There were a number of possible naming styles and, in the end,
it is the module writer's perogative.
The i-prefix is supposed to be a reminder that the functions return
iterators. It has a nice, snappy feel to it and is unapologetic about
any resemblance to names used in Apple/MacIntosh marketing.
The x prefixes all predate generators and iterators. They are associated
with deprecated things like xreadlines. X is used to mark errors.
X-men are mutants. Ex-girlfriends are not positive either.
My only misgiving about the naming convention is that
islice() can be read as is-lice instead of i-slice. Oh, well.
> def xgroup(iter,n=2):
> r"""Iterate `n`-wise (default pairwise) over `iter`.
Sounds interesting enough to put on the list of ideas being
entertained. Also, it is consistent with the idea of "iterator
algebra".
For the time being, my goal is to start with a small set of
tools that have proven useful in Haskell and SML. Over
time, other candidates will considered.
This one would have a much better chance if some common
use cases could be shown and if it can be demonstrated that
it is a useful building block in combination with the other
itertools.
> def xwindow(iter, n=2, s=1):
> r"""Move an `n`-item (default 2) windows `s` steps (default 1) at a
time
> over `iter`.
I've needed this one only one time in twenty-five years of programming
(for the markov algorithm). Again, solid use cases are needed.
> A last thing, I think a `cycle` generator would be an really useful idiom
to
> have,
It has proved useful in the SML world.
I'm not sure it makes sense to implement it in C though.
Raymond Hettinger
Paul Rubin
> > Wouldn't `xfilter`, `xmap` etc. be a better naming convention than
> > `ifilter`, `imap` etc?
>
> There were a number of possible naming styles and, in the end,
> it is the module writer's perogative.
>
> The i-prefix is supposed to be a reminder that the functions return
> iterators. It has a nice, snappy feel to it and is unapologetic about
> any resemblance to names used in Apple/MacIntosh marketing.
imap is also an email transfer protocol. A python program I'm hacking
on now has an imap class that implementst that protocol. I guess I'll
have to rename it.
I think xrange, etc. are already a convention for the closest thing
Python has traditionally had to iterators. Maybe those functions
should just be updated to make ordinary iterators.
Jimmy Retzlaff
> "Alexander Schmolck"
...
>> def xwindow(iter, n=2, s=1):
>> r"""Move an `n`-item (default 2) windows `s` steps (default 1) at
>>a time
>> over `iter`.
>
>I've needed this one only one time in twenty-five years of programming
>(for the markov algorithm). Again, solid use cases are needed.
...
I've used something like this several times in recent weeks when doing
some simple computational geometry. One way to store a polygon is as a
sequence of points that represent the vertices of the polygon. If you
want to perform some operation on the edges of the polygon, an xwindow
style iterator can be used to iterate over consecutive pairs of points.
One nuance in this domain is that sometimes polygons are represented as:
[p0, p1, p2]
where each p is a distinct point. But at other times the starting point
is repeated at the end for various reasons:
[p0, p1, p2, p0]
xwindow as shown above would do just fine when the starting point is
repeated (i.e., it would yield all the edges). But in the first
representation above, either an option would be needed to tell the
iterator to "wrap" around until the beginning of the "window" has gone
through all the elements of xwindow's parameter, or it would have to be
used in a manner similar to:
for edge in xwindow(polygon + [polygon[0]]):
if thisisthebleeding(edge):
print 'this is new stuff'
This can be ugly if the polygon is large (one of the motivations for
using iterators in the first place). Of course, something like itercat
from http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66448 could
be used here. Come to think of it, I'd probably use itercat even more
often than xwindow. I can see how following use cases like this could
lead to the size of itertools exploding pretty quickly.
Jimmy
Chad Netzer
> For the time being, my goal is to start with a small set of
> tools that have proven useful in Haskell and SML. Over
> time, other candidates will considered.
Would you be open to having an irange() call, that could be used instead
of xrange(), and would be able to return long integers (perhaps only
when appropriate)?
I've been (very casually) working on updating range() and xrange() to
work with Python long ints. With xrange() it is tricky because I want
it to return the xrange() proxy, for backwards compatibility and speed,
except when longs are required. My best option up to this point has
just been to make an xlrange() generator and leave xrange() alone. This
could be an irange() instead, and range() could then easily be extended
(and xrange() left as is or deprecated).
This may sound like a pointless exercise (meant for language
'consistency' and having little practical value), but I've encountered
several bugs where people assume it will work, when doing things like:
range( long(1/small_float_value), long(1/another_small_float_value) )
Ultimately, I'd be happy if both of these worked:
a = long(1/1e-17) # 100000000000000000L
range(a, a+10)
irange(a, a+10)
My current solution would be slightly uglier (IMO):
range(a, a+10)
xlrange(a, a+10)
If you'd accept an irange() patch, I'll send you one by next week...
(and if you accept it, I'll send a range() patch to sourceforge)
Raymond Hettinger
> Come to think of it, I'd probably use itercat even more
> often than xwindow.
If I get a few more requests for it, itercat() will go in.
In the docs, I mention that this tool is the current
most likely addition to the module. One proposed
name for the function is chain().
> I can see how following use cases like this could
> lead to the size of itertools exploding pretty quickly.
You're absolutely right.
Raymond Hettinger
SUZUKI Hisao
> 2.3. Much improved since the first alpha, chockfull of things you'd
> like to check out:
>
> http://www.python.org/2.3/
If you install it cleanly on Windows in Japanese locale, you
cannot even start up the IDLE. Perhaps you cannot do it also in
Chinese and Korean locales.
In that case, just put the following as
C:\Python23\Lib\site-packages\sitecustomize.py
----------------------------------------------------------------
# On Windows, some default encodings are not provided
# by Python (e.g. "cp932" in Japanese locale), while they
# are always available as "mbcs" in each locale.
# Ensure them usable by aliasing to "mbcs" in such case.
import sys
if sys.platform == 'win32':
import locale, codecs
enc = locale.getdefaultlocale()[1]
if enc.startswith('cp'): # "cp***" ?
try:
codecs.lookup(enc)
except LookupError:
import encodings
encodings._cache[enc] = encodings._unknown
encodings.aliases.aliases[enc] = 'mbcs'
----------------------------------------------------------------
(I have already posted the patch with a bit of different flavor
to sf.net as Request ID 671666 on 21 Jan 2003; however, it seems
to have been simply ignored. I am afraid the problem may be
critical to many pythoneers in East Asia.)
-- SUZUKI Hisao
Raymond Hettinger
> of xrange(), and would be able to return long integers (perhaps only
> when appropriate)?
I recommend addressing the problem directly and posting a feature
request or patch for changes to range() or xrange().
Raymond Hettinger
Erik Max Francis
> The x prefixes all predate generators and iterators. They are
> associated
> with deprecated things like xreadlines. X is used to mark errors.
> X-men are mutants. Ex-girlfriends are not positive either.
Or, more concretely :-), the x prefix tends to indicate, "This has the
same functionality as the non-prefixed version, but potentially saves
memory by encapsulating the work in a proxy object."
--
Erik Max Francis / m...@alcyone.com / http://www.alcyone.com/max/
__ San Jose, CA, USA / 37 20 N 121 53 W / &tSftDotIotE
/ \ Success and failure are equally disastrous.
\__/ Tennessee Williams
Polly Wanna Cracka? / http://www.pollywannacracka.com/
The Internet resource for interracial relationships.
Tim Peters
> ...
> (I have already posted the patch with a bit of different flavor
> to sf.net as Request ID 671666 on 21 Jan 2003;
For easier browsing:
http://www.python.org/sf/671666
> however, it seems to have been simply ignored.
It clearly wasn't ignored: Martin rejected the patch. Unfortunately, he
left for a long vacation during the week that elapsed before you responded
to his rejection. He's still on vacation.
> I am afraid the problem may be critical to many pythoneers in East
> Asia.)
It would probably help if even one of them added a comment to the patch
report <wink>. I don't understand the issues, and defer to Martin based on
his superb track record in resolving these kinds of issues. There are still
a few months to resolve this before 2.3 final is released, so don't worry
too much.
Chad Netzer
Right. I'll finish up work on the range() patch for 2.3, and submit.
The changes required to support the long integers for xrange() have been
discussed in the past, and are hairy. I never bothered finishing, as
the changes were just too invasive (and therefore unlikely to be
accepted for what is arguably a small gain for most).
An xlrange() in pure python is easy to implement, and I'll work on a
C-based irange() for consideration in itertools. itertools at least
provides a sensible namespace for it (though I'd understand if you
considered it pollution, and rejected it as such.:) I'll try to state
my case with simple, maintainable code before any further debate on the
matter. Regardless, I think the range() change should only be accepted
into the core if an xlrange() type functionality (whether it is called
xlrange(), irange(), etc.) is also made available, simply for
consistency.
John Luebs
> Guido van Rossum:
>>- Extended slices, e.g. "hello"[::-1] returns "olleh".
>
> This is a troll, right? :-)
You wish.
The third parameter is the step parameter.
So 'hello'[::1] returns 'hello'. 'hello'[::2] returns 'hlo'. A negative
step value is interpreted as you would expect (step left, instead of
right), but starting from the right of the sequence. So [::-1] will
reverse, as shown above.
This syntax has been valid for a while. In fact you can try it with an
older version of Python, with the Numeric package. Initialize an array and
then take the array and do a[::-1].
Python 2.3 simply adds support for this slice feature in the builtin sequence
types.
Greg Ewing (using news.cis.dfn.de)
> My only misgiving about the naming convention is that
> islice() can be read as is-lice instead of i-slice.
No, you'll have to import that from the new bug-detecting
module being developed for 2.4, which it is rumoured will
also provide isaphid(), istermite(), isvarroamite() and
ispaintedapplemoth().
--
Greg Ewing, Computer Science Dept,
University of Canterbury,
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg
Alex Martelli
> "Raymond Hettinger" <vze4...@verizon.net> writes:
>> > Wouldn't `xfilter`, `xmap` etc. be a better naming convention than
>> > `ifilter`, `imap` etc?
>>
>> There were a number of possible naming styles and, in the end,
>> it is the module writer's perogative.
>>
>> The i-prefix is supposed to be a reminder that the functions return
>> iterators. It has a nice, snappy feel to it and is unapologetic about
>> any resemblance to names used in Apple/MacIntosh marketing.
>
> imap is also an email transfer protocol. A python program I'm hacking
> on now has an imap class that implementst that protocol. I guess I'll
> have to rename it.
No need, thanks to namespaces (a honkin' great idea). itertools.imap
indicates one thing, yourmodule.imap indicates a completely different
one, and you both live happily ever after. I _would_ suggest adhering
to the convention that class names are capitalized, but that's got
nothing to do with "nameclashes".
> I think xrange, etc. are already a convention for the closest thing
> Python has traditionally had to iterators. Maybe those functions
> should just be updated to make ordinary iterators.
Not without breaking, e.g., xrange(23, 17923, 12)[15] -- which now
has a perfectly good result of 203 and would break if xrange
returned an iterator (or any other non-indexable object).
Alex
Alexander Schmolck
>
> > I think xrange, etc. are already a convention for the closest thing
> > Python has traditionally had to iterators. Maybe those functions
> > should just be updated to make ordinary iterators.
>
> Not without breaking, e.g., xrange(23, 17923, 12)[15] -- which now
> has a perfectly good result of 203 and would break if xrange
> returned an iterator (or any other non-indexable object).
It could always return and indexable iterator.
alex
Alexander Schmolck
> > Wouldn't `xfilter`, `xmap` etc. be a better naming convention than
> > `ifilter`, `imap` etc?
>
> There were a number of possible naming styles and, in the end,
> it is the module writer's perogative.
Point taken, but your itertools module is somewhat special in that it
effectively creates a naming convention for the *whole* language. It is
sometimes necessary to explicitly mark something as a generator or equivalent
(e.g. if there is also an equivalent sequence version) so there should be a
common way to do so and the precedent is set by your itertools module as the
first sanctified (general purpose) generator module.
So if you adhere to your naming scheme then I (and I guess others) also have
to adopt it (and change code). This in itself is unavoidable and I wouldn't
complain, if I hadn't the impression that a) there already is a (if maybe not
fully established) convention for `x` to denote 'lazy' and 'i' to denote
'inplace' and b) there should be as few such strange prefixes as possible and
c) just using 'i' for both leads to ambiguity.
For example I've got a (reasonably big) matrix library. Obviously, it is
necessary to provide facilities to do operations inplace for efficiency
reasons. So there is an `add` as well as an `iadd`, analogous to `__add__` and
`__iadd__` and similarly for other functions such as `sqrt` (which BTW
demonstrates how overused 'i' already is, as `isqrt` might as well mean
'complex squareroot'). I can't really see how else I could call the inplace
versions.
As an example how just prefixing 'i' for both 'inplace' and 'iterator' is
confusing: I've currently got a function called `iconcat` to destructively add
further sequences to the first argument, but it might as well denote what you
consider including as `chain`.
> X-men are mutants. Ex-girlfriends are not positive either.
If you really don't like 'x', how about 'g' -- at least that isn't as
overloaded as 'i' (iterator, inplace, imaginary, interactive ...)?
>
> This one would have a much better chance if some common
> use cases could be shown and if it can be demonstrated that
> it is a useful building block in combination with the other
> itertools.
Sure, I'm all for including conservatively and I really like the selection
you've made so far and, apart from the i prefix, the descriptive names you've
chosen (the only quibble is `ifilterfalse` -- it is not difficult to express
with `ifilter` and I find the name a bit opaque; how about
`(x/i)reject`?). The thing I'd most like to see included would be an `i/xcat`
or `chain` method, so add my vote for that. Anyway, before I'll give use cases
for xwindow below, how about collecting potential candidates (with python
implementation) on a wiki page?
>
>
> > def xwindow(iter, n=2, s=1):
> > r"""Move an `n`-item (default 2) windows `s` steps (default 1) at a
> time
> > over `iter`.
>
> I've needed this one only one time in twenty-five years of programming
> (for the markov algorithm). Again, solid use cases are needed.
I think iterating over direct neighbours is not uncommon. One case already
mentioned would to create edges out of vertices, but there are many
others. For example, I've used the xwindow construct for backward-forward
feature selection (which is analogous to the vertex case) and to created a
CamelCaseString out of a sequence of strings (which is, I guess similar to the
use of context information you made).
I can go into more detail, but maybe this gives the idea:
# last endpoint is next starting point
for (start, end) in xwindow(vertices):
f(start,end)
...
# context information
for (this, next) in xwindow(l):
if relationship(this, next): ...
else: ...
Admittedly, you could already express this common case:
xwindow(l)
using only 2 of your itertools:
izip(l, islice(l,1))
but it is more verbose, a bit less efficient and I guess not as intuitive as
moving a window over a sequence.
>
> > A last thing, I think a `cycle` generator would be an really useful idiom
> to
> > have,
>
> It has proved useful in the SML world.
> I'm not sure it makes sense to implement it in C though.
Well, I don't know how much faster it would be, so I can't say. Even if there
isn't a compelling reason to implement it in C for speed the fact that it is
quite useful and logically belongs with the rest of itertools would be reason
enough, IMHO.
even simple cases such as
for line, color in zip(lines, cycle('red', 'green', 'blue')):
draw(line, color)
seem much nicer to me than:
colors = 'red', 'green', 'blue'
for i, line in enumerate(lines):
draw(line, colors[i % len(colors)])
alex
Alexander Schmolck
>
> Admittedly, you could already express this common case:
>
> xwindow(l)
>
> using only 2 of your itertools:
>
> izip(l, islice(l,1))
Ooops, I guess that would need to be:
izip(l, islice(l,1,None))
The nongenerator version is:
zip(l, l[1:])
Maybe a 'drop'/'take' as in
izip(l, drop(1,l))
would be handy, too?
alex
Christian Tanzer
"Raymond Hettinger" <pyt...@rcn.com> wrote:
> > def xgroup(iter,n=2):
> > r"""Iterate `n`-wise (default pairwise) over `iter`.
>
> Sounds interesting enough to put on the list of ideas being
> entertained. Also, it is consistent with the idea of "iterator
> algebra".
>
> For the time being, my goal is to start with a small set of
> tools that have proven useful in Haskell and SML. Over
> time, other candidates will considered.
>
> This one would have a much better chance if some common
> use cases could be shown and if it can be demonstrated that
> it is a useful building block in combination with the other
> itertools.
I happen to use this function. Greping over my source tree for
`pairwise` shows 46 uses in 1074 files (disclaimer: that is
a usage frequency of only 0.00017 <wink>).
Typical usage (blatantly made up):
def test_sort(some_list) :
sorted_list = myq_sort(some_list)
for l, r in pairwise(sorted_list) :
assert l <= r
I never needed anything like `triplewise`, though.
--
Christian Tanzer tan...@swing.co.at
Glasauergasse 32 Tel: +43 1 876 62 36
A-1130 Vienna, Austria Fax: +43 1 877 66 92
Steven Taschuk
> "Alexander Schmolck"
[...]
> > def xwindow(iter, n=2, s=1):
> > r"""Move an `n`-item (default 2) windows `s` steps (default 1) at a
> time
> > over `iter`.
>
> I've needed this one only one time in twenty-five years of programming
> (for the markov algorithm). Again, solid use cases are needed.
Schmolck's xgroup(it, n) is xwindow(it, n, n).
More seriously:
Go pairwise to compute a sequence's first differences (for its own
sake, or to, say, approximate a function's derivative or compute
its Newton series).
Generalizing, go k-wise to compute g(n) = f(n) - f(n-k). (Some
stock analysts use this as an indicator; they call it "momentum",
I think. They also use g(n) = f(n)/f(n-k).)
Go pairwise on a token stream to get one-token lookahead in a
parser. (This one's my favourite. But see below re "fading".)
Go 3-wise to smooth f(0), f(1), ... by replacing f(x) with the
average of f(x-1), f(x), and f(x+1).
Go 3-wise to implement Euler's sequence accelerator:
t(n) = s(n+1) - (s(n+1) - s(n))**2/(s(n-1) - 2*s(n) + s(n+1))
Go 3-wise to compute linear automata in which the next state of a
cell depends on its neighbours' states.
These can be summarized by saying that going k-wise with step 1
implements a FIFO buffer k objects wide. I don't think the
usefulness of such buffers is controversial, though it's certainly
open to question whether xwindow() would be the best way to
implement them.
All of the above (except for implementing xgroup() in terms of
xwindow()) use a step of 1. I can only think of one case where a
step other than 1 or the window size would be useful: drawing a
Bezier spline on a polyline by first going pairwise to insert
midpoints, then going 3-wise with step 2 for the actual spline
computation. But this is dubious as a use case, since it's pretty
much as easy, and probably clearer, to go 3-wise with step 1 on
the original data. (More complicated splines than the one I have
in mind might do better.)
So perhaps the step argument is unnecessary.
Note also that these use cases differ in their desired behaviour
at the end of the sequence:
- Most want the window to stop when its leading edge goes past the
end of the underlying data.
- The parser with lookahead wants the window to "fade out", stopping
only when its *trailing* edge goes past the end of the underlying
data.
- A smoothing algorithm might want either of the above.
- As Ratzleff pointed out, some use cases involving polygons want
the leading edge to cycle back to the beginning of the sequence,
stopping when the trailing edge reaches the end of the sequence.
The same point might be made for the beginning of the sequence: a
smoothing algorithm that wants fade-out might also want fade-in.
When fading out (in), there's also the question of whether the
window ought to shrink (grow) or be padded out with, say, None.
One interpretation of this embarrassment of variation is that
xwindow() is an over-generalization.
--
Steven Taschuk 7\ 7'Z {&~ .
stas...@telusplanet.net Y r --/hG-
(__/ )_ 1^1`
Alex Martelli
Yes, it's not forbidden for an iterator object to be also
indexable -- weird (but then some would say xrange objects
already ARE weird, I guess) but not forbidden. Similarly,
len(xrange(7)) would have to keep returning 7, etc.
So what would be the added value of forcing:
x = xrange(n)
assert x is iter(x)
to be true? It would only seem to break such usage as:
N_times = xrange(N)
for i in N_times:
for j in N_times:
process(i, j)
What advantage would you think breaking this usage would
confer? Or, if you agree that breaking this usage is a
DIS-advantage (albeit not a huge one), what would in your
opinion be the compensating advantages? I just cannot see
any -- but it is, of course, quite possible that I'm just
having a blind spot, so, please explain, thanks!
Alex
Alexander Schmolck
> Alexander Schmolck wrote:
>
> > Alex Martelli <al...@aleax.it> writes:
> >> Not without breaking, e.g., xrange(23, 17923, 12)[15] -- which now
> >> has a perfectly good result of 203 and would break if xrange
> >> returned an iterator (or any other non-indexable object).
> >
> > It could always return and indexable iterator.
>
> It would only seem to break such usage as:
>
> N_times = xrange(N)
>
> for i in N_times:
> for j in N_times:
> process(i, j)
>
Excellent point. So it is not an option.
alex
Paul Rubin
> > >> Not without breaking, e.g., xrange(23, 17923, 12)[15] -- which now
> > >> has a perfectly good result of 203 and would break if xrange
> > >> returned an iterator (or any other non-indexable object).
> > >
> > > It could always return and indexable iterator.
> >
> > It would only seem to break such usage as:
> >
> > N_times = xrange(N)
> >
> > for i in N_times:
> > for j in N_times:
> > process(i, j)
>
> Excellent point. So it is not an option.
Yuck, I didn't you could do that with xrange. It's surprising and
somewhat bizarre.
Alexander Schmolck
Well, I wouldn't disagree that xrange is a bit of a freak (BTW can anyone tell
me why there is a need for both `slice` and `xrange`?) but given that xrange
was intended as a lazy equivalent to range, this behavior maybe shouldn't be
too surprising (if N_times where just a normal list, generated by range, then
clearly it would still have the same semantics).
alex
Erik Max Francis
> Yuck, I didn't you could do that with xrange. It's surprising and
> somewhat bizarre.
Neither did I, although abstractly it makes sense. I doubt that this
behavior is relied upon very much, though.
--
Erik Max Francis / m...@alcyone.com / http://www.alcyone.com/max/
__ San Jose, CA, USA / 37 20 N 121 53 W / &tSftDotIotE
/ \ There are defeats more triumphant than victories.
\__/ Montaigne
Python chess module / http://www.alcyone.com/pyos/chess/
A chess game adjudicator in Python.
Magnus Lie Hetland
[snip]
>At least there is now an izip() in itertools (although, again, I would
>prefer the 'x' naming, or the addition of irange() and the deprecation
>of xrange())
... and xreadlines()...
--
Magnus Lie Hetland "Nothing shocks me. I'm a scientist."
http://hetland.org -- Indiana Jones
Magnus Lie Hetland
>> def xgroup(iter,n=2):
>> r"""Iterate `n`-wise (default pairwise) over `iter`.
>> def xwindow(iter, n=2, s=1):
>> r"""Move an `n`-item (default 2) windows `s` steps (default 1) at a
>time
>> over `iter`.
>
>I've needed this one only one time in twenty-five years of programming
>(for the markov algorithm). Again, solid use cases are needed.
I use this sort of thing a lot, when discretizing/processing time
series -- a sliding window is useful for many things there (e.g.
running averages). At the moment I'm using numarray for this stuff,
but wouldn't mind an iterator approach -- especially with some
standard library tools :)
Jp Calderone
> Paul Rubin wrote:
>
> > Yuck, I didn't you could do that with xrange. It's surprising and
> > somewhat bizarre.
>
> Neither did I, although abstractly it makes sense. I doubt that this
> behavior is relied upon very much, though.
>
I use it a lot. Create one xrange object as a module attribute, then
iterate over that single object anywhere it is needed. Cuts the need for a
function call, and an allocation/deallocation pair.
Jp
--
http://catandgirl.com/view.cgi?44
--
up 13 days, 8:28, 4 users, load average: 0.02, 0.03, 0.00
Christos TZOTZIOY
(Christian Tanzer) might have written:
>I never needed anything like `triplewise`, though.
That probably means you have a happy marriage (or long-term relationship
:)
--
TZOTZIOY, I speak England very best,
bo...@sil-tec.gr
(I'm a postmaster luring spammers; please spam me!
...and my users won't ever see your messages again...)
Michael Hudson
> Raymond Hettinger wrote:
> > My only misgiving about the naming convention is that
> > islice() can be read as is-lice instead of i-slice.
>
> No, you'll have to import that from the new bug-detecting
> module being developed for 2.4, which it is rumoured will
> also provide isaphid(), istermite(), isvarroamite() and
> ispaintedapplemoth().
I comprehensively misread this post and thought one of those was
"isterminating()" :-)
Cheers,
M.
--
Every day I send overnight packages filled with rabid weasels to
people who use frames for no good reason.
-- The Usenet Oracle, Oracularity #1017-1