is parameter an iterable?
py
to be an iterable (something i can loop over)...so I dont care if its a
list, tuple, etc. So I need a way to make sure that the argument is an
iterable before using it. I know I could do...
def foo(inputVal):
if isinstance(inputVal, (list, tuple)):
for val in inputVal:
# do stuff
...however I want to cover any iterable since i just need to loop over
it.
any suggestions?
py
Dan Sommers
"py" <code...@gmail.com> wrote:
> any suggestions?
Just do it. If one of foo's callers passes in a non-iterable, foo will
raise an exception, and you'll catch it during testing. Watch out for
strings, though:
>>> def foo(i):
... for j in i:
... print j
>>> foo([1, 3, 4, 5, 6, 7])
1
3
4
5
6
7
>>> foo("hello")
h
e
l
l
o
Regards,
Dan
--
Dan Sommers
<http://www.tombstonezero.net/dan/>
marduk
You could probably get away with
if hasattr(inputVal, '__getitem__')
py
> Just do it. If one of foo's callers passes in a non-iterable, foo will
> raise an exception, and you'll catch it during testing
That's exactly what I don't want. I don't want an exception, instead I
want to check to see if it's an iterable....if it is continue, if not
return an error code. I can't catch it during testing since this is
going to be used by other people.
Thanks for the suggestion though.
Jean-Paul Calderone
>Dan Sommers wrote:
>> Just do it. If one of foo's callers passes in a non-iterable, foo will
>> raise an exception, and you'll catch it during testing
>
>That's exactly what I don't want. I don't want an exception, instead I
>want to check to see if it's an iterable....if it is continue, if not
>return an error code.
Error codes are not the common way to do things in Python. Exceptions are. There's generally no reason to avoid exceptions. Error codes allow errors to pass silently, which leads to bugs that nobody notices for long periods of time.
You should let the exception be raised. You shouldn't try to return an error code.
>I can't catch it during testing since this is going to be used by
>other people.
Then they'll catch it during their testing >:) If you return an error code instead, they are just as likely to pass in bad data, and even *more* likely to not see that an error has occurred, causing their programs to be incorrect.
Jean-Paul
Rocco Moretti
No, you probably couldn't.
##################
>>> def g(s):
for i in xrange(s):
yield i+s
>>> m = g(5)
>>> hasattr(m, '__getitem__')
False
###################
I'd do something like:
#####################
def foo(inputVal):
try:
iter(inputVal) # Can you change it into an interator?
except TypeError:
# Return Error Code
else:
for val in inputVal:
# do stuff
#######################
Again, you'll have to be careful about strings.
Robert Kern
> Dan Sommers wrote:
>
>>Just do it. If one of foo's callers passes in a non-iterable, foo will
>>raise an exception, and you'll catch it during testing
>
> That's exactly what I don't want. I don't want an exception, instead I
> want to check to see if it's an iterable....if it is continue, if not
> return an error code.
Why return an error code? Just pass along the exception (i.e. do nothing
special). Python's exception mechanism is far superior to error codes.
Don't try to fight the language.
> I can't catch it during testing since this is
> going to be used by other people.
Then *they'll* catch it during testing.
--
Robert Kern
rk...@ucsd.edu
"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
py
suggestion...let the exception be raised.
Thanks.
Carl Friedrich Bolz
Note that using error codes is usually quite "unpythonic", the way to
signal that something is exceptional (not necessarily wrong) is, well,
an exception.
Anyway, one way to solve this is the following:
def foo(input_val):
try:
iterator = iter(input_val)
except TypeError:
# do non-iterable stuff
else:
for val in iterator:
# do loop stuff
Cheers,
Carl Friedrich Bolz
Roy Smith
py <code...@gmail.com> wrote:
>I have function which takes an argument. My code needs that argument
>to be an iterable (something i can loop over)...so I dont care if its a
>list, tuple, etc.
My first thought was to just write your loop inside a try block and
catch the error if it wasn't iterable, but then I noticed that you get:
TypeError: iteration over non-sequence
I was kind of hoping for a more specific exception than TypeError.
You can't tell the difference between:
try:
for i in 5:
print i + 1
except TypeError:
print "non-iterable"
and
try:
for i in ["one", "two", "three"]:
print i + 1
except TypeError:
print "can't add string and integer"
Unfortunately, you can't just try it in a bodyless loop to prove that
you can iterate before doing the real thing because not all iterators
are idempotent.
It's an interesting problem.
Grant Edwards
If I were those other people, and you decided to return error
codes to me instead of passing up the proper exception (the
good, Pythonic thing to do), I'd be fairly pissed off at you.
An exception is the _right_ way to let the caller know
something is wrong.
--
Grant Edwards grante Yow! I smell like a wet
at reducing clinic on Columbus
visi.com Day!
Dave Hansen
<code...@gmail.com> wrote:
Then catch the exception yourself.
>>> def foo2(i):
try:
for j in i:
print j
print "Success!"
return 0
except TypeError, e:
print "Bad foo. No donut.", e
return -1
>>> joe = foo2([1,3,5,7,9])
1
3
5
7
9
Success!
>>> print joe
0
>>> bob = foo2(2)
Bad foo. No donut. iteration over non-sequence
>>> print bob
-1
>>>
Regards,
-=Dave
--
Change is inevitable, progress is not.
lmay...@gmail.com
>>> import types
>>> def foo(inputVal):
inValType = type(inputVal)
if inValType==types.ListType or inValType==types.TupleType:
for val in inputVal:
print val
else:
print 'Wrong input Type'
>>> list = [1,2,3,4]
>>> foo(list)
1
2
3
4
>>> tup = ('a', 'b', 'c')
>>> foo(tup)
a
b
c
>>> foo(9)
Wrong input Type
>>>
Steven D'Aprano
> On 15 Nov 2005 11:01:48 -0800,
> "py" <code...@gmail.com> wrote:
>
>> I have function which takes an argument. My code needs that argument
>> to be an iterable (something i can loop over)...so I dont care if its a
>> list, tuple, etc. So I need a way to make sure that the argument is an
>> iterable before using it. I know I could do...
...
> Just do it. If one of foo's callers passes in a non-iterable, foo will
> raise an exception, and you'll catch it during testing.
It isn't during testing so much that he needs to watch out, as run-time.
You have three options:
(1) The "if the user is silly enough to pass a non-iterable to my
function, they deserve to have it fail" school of thought. Dan's advise
comes under this heading.
(2) The "if the user passes a non-iterable, I want to catch the exception
and recover gracefully" school of thought. Recovering gracefully may mean
raising your own exception, with a user friendly error message ("Hey
butthead, pass an iterable willya!!!"), or it may mean just catching the
exception and doing something else, depending on the needs of your
program. (For example, as a user-friendly feature, you might want to treat
ints as if they were iterables of one item only, so users can pass 5 as an
argument instead of [5].)
For the second case, what you want to do is wrap your code in a
try...except block and catch the exception raised when a non-iterator is
passed as an argument.
Which exception is that? I leave that as an exercise for the reader.
(Hint: Just Do It and read the traceback Python prints.)
Actually, I think "Just Do It" might make a good motto for Python.
--
Steven
DonQuixote...@gmx.net
Regards,
Karsten.
Ben Finney
> Maybe this helps:
>
> >>> import types
> >>> def foo(inputVal):
> inValType = type(inputVal)
> if inValType==types.ListType or inValType==types.TupleType:
And what of user-created types that are iterable?
What of user-created iterable types that don't inherit from any of the
built-in iterable types?
--
\ "A good politician is quite as unthinkable as an honest |
`\ burglar." -- Henry L. Mencken |
_o__) |
Ben Finney
Steven D'Aprano
That would be the Look Before You Leap school of thought.
That is rarely the best way of doing things in Python. (Note I said
*rarely*, not *never*.) If you insist, you would have to do something like
this:
def is_iterable(obj):
"""Returns True if obj is *probably* an iterable and False
otherwise. Not that some weird custom objects that break the
rules may give incorrect results.
"""
if type(obj) in (type([]), type(()), type("")):
return True
elif hasattr(obj, next) and callable(obj.next):
return True
elif hasattr(obj, __getitem__):
return True
else:
# I don't think I missed anything...
return False
But in fact there is an easier way:
def is_iterable(obj):
"""Just Do It."""
try:
for item in obj:
break
except TypeError:
return False
return True
That works, but it has a side-effect: if you pass it an iterable that gets
consumed, you now have used up the first item of it (e.g. if you a
reading lines from a file, you've just used the first line and won't get
it back unless you reset the file pointer).
So do your test inside your code:
def foo(inputVal):
try:
for val in inputVal:
# do stuff
except TypeError, msg:
if msg == "iteration over non-sequence":
# handle non-iterable case
else:
# some other TypeError is a bug, so re-raise the exception
raise
Alex Martelli
...
> TypeError: iteration over non-sequence
>
> I was kind of hoping for a more specific exception than TypeError.
> You can't tell the difference between:
>
> try:
> for i in 5:
> print i + 1
> except TypeError:
> print "non-iterable"
>
> and
>
> try:
> for i in ["one", "two", "three"]:
> print i + 1
> except TypeError:
> print "can't add string and integer"
>
> Unfortunately, you can't just try it in a bodyless loop to prove that
> you can iterate before doing the real thing because not all iterators
> are idempotent.
It's not hard...:
try:
_it = iter(whatever)
except TypeError:
print 'non-iterable'
else:
for i in _it: # etc, etc
Alex
Steven D'Aprano
> You can't tell the difference between:
>
> try:
> for i in 5:
> print i + 1
> except TypeError:
> print "non-iterable"
>
> and
>
> try:
> for i in ["one", "two", "three"]:
> print i + 1
> except TypeError:
> print "can't add string and integer"
try:
for item in obj:
do_stuff(item)
except TypeError, msg:
if msg == "iteration over non-sequence":
handle_non_iterator()
else:
# re-raise the exception
raise
--
Steven.
Steven D'Aprano
> Maybe this helps:
Unfortunately, it doesn't. If you read the sample code
the original poster first gave, you will see that Py's
example is virtually the same as yours:
[original code]
def foo(inputVal):
if isinstance(inputVal, (list, tuple)):
for val in inputVal:
# do stuff
It uses the same program logic, only the implementation
is different: your code tests the object's type, and
compares it to the type of list and tuple. Py's code
tests the object's type, comparing it to lists and tuples.
Type testing gives less information than using
isinstance, and takes more work to do it. For instance,
Py's code will correctly work with subclasses of lists,
yours will wrongly reject them.
Check Py's requirements carefully:
"I have function which takes an argument. My code
needs that argument to be an iterable (something i can
loop over)...so I dont care if its a list, tuple, etc."
Notice the "etc"? There are lots of iterables other
than lists and tuples. There are xrange objects, files,
strings, buffers, sub-classes of all of those, classes
that implement __getitem__(), classes that implement
next(), and probably more things that I have forgotten.
Your code doesn't check for any of those.
--
Steven.
Dan Sommers
Steven D'Aprano <st...@REMOVEMEcyber.com.au> wrote:
> try:
> for item in obj:
> do_stuff(item)
> except TypeError, msg:
> if msg == "iteration over non-sequence":
> handle_non_iterator()
> else:
> # re-raise the exception
> raise
But what it do_stuff tries to iterate over a non-sequence?
Roy Smith
Steven D'Aprano <st...@REMOVEMEcyber.com.au> wrote:
> try:
> for item in obj:
> do_stuff(item)
> except TypeError, msg:
> if msg == "iteration over non-sequence":
> handle_non_iterator()
> else:
> # re-raise the exception
> raise
That's the obvious solution, but it's a poor one because it depends on an
undocumented feature of the language. What's documented is that TypeError
is raised; it's not documented what the text of the message will be.
Steven D'Aprano
> On Wed, 16 Nov 2005 13:12:38 +1100,
> Steven D'Aprano <st...@REMOVEMEcyber.com.au> wrote:
>
>> try:
>> for item in obj:
>> do_stuff(item)
>> except TypeError, msg:
>> if msg == "iteration over non-sequence":
>> handle_non_iterator()
>> else:
>> # re-raise the exception
>> raise
>
> But what it do_stuff tries to iterate over a non-sequence?
I can think of two solutions:
(1) Write do_stuff carefully, and test it thoroughly on its own. If there
are no bugs in do_stuff, you can be sure it isn't going to try to iterate
over a non-iterator. This is a good argument for writing small,
encapsulated, easily analysed and debugged pieces of code.
(2) Muck about writing fragile, possibly buggy code:
# untested
try:
for item in obj:
flag = False
try:
do_stuff(item)
except TypeError, msg:
if msg == "iteration over non-sequence":
flag = True
# re-raise the exception
raise
except TypeError, msg:
if flag:
raise
if msg == "iteration over non-sequence":
handle_non_iterator()
else:
raise
I think you can guess where my preference lies.
--
Steven.
Steven D'Aprano
It would be nice if The Powers That Be would document the specific error
messages, so that we could rely on them in total safety.
But even without that, the consequences aren't especially dire. What
happens if the error message changes in some future version of Python?
Then the error won't be caught, the exception will be re-raised, and your
testing will catch it immediately.
It isn't the ideal solution, but it is a solution.
--
Steven.
Rick Wotnaz
news:pan.2005.11.15....@REMOVETHIScyber.com.au:
> def foo(inputVal):
> try:
> for val in inputVal:
> # do stuff
> except TypeError, msg:
> if msg == "iteration over non-sequence":
> # handle non-iterable case
> else:
> # some other TypeError is a bug, so re-raise the
> exception raise
Does this in fact work on your system? On mine (2.4.1 (#65, Mar 30
2005, 09:13:57) [MSC v.1310 32 bit (Intel)]), it doesn't seem to. I
tried
if msg.find("iteration over non-sequence") >= 0:
... but I got a traceback, and
AttributeError: TypeError instance has no attribute 'find'
... which leads me to belive that 'msg' is not type(str). It can be
coerced (str(msg).find works as expected). But what exactly is msg?
It appears to be of <type 'instance'>, and does not test equal to a
string. This is not the least surprise to me.
--
rzed
Roy Smith
Rick Wotnaz <des...@wtf.com> wrote:
It's an easy experiment to do:
-------------------
Roy-Smiths-Computer:play$ cat ex.py
#!/usr/bin/env python
try:
1 + "foo"
except TypeError, msg:
print type(msg)
print msg
print repr(msg)
print dir(msg)
Roy-Smiths-Computer:play$ py ex.py
<type 'instance'>
unsupported operand type(s) for +: 'int' and 'str'
<exceptions.TypeError instance at 0x36d968>
['__doc__', '__getitem__', '__init__', '__module__', '__str__', 'args']
---------------------
Fredrik Lundh
> ... which leads me to belive that 'msg' is not type(str). It can be
> coerced (str(msg).find works as expected). But what exactly is msg?
> It appears to be of <type 'instance'>, and does not test equal to a
> string.
it's an instance of the exception type, of course.
:::
if you do
raise SomeError, value
Python will actually do
raise SomeError(value)
(that is, create a SomeError exception and pass the value as its
first argument).
you can use either form in your code (I prefer the latter myself).
:::
as for catching the exceptions, if you do
try:
...
except SomeError, v:
...
Python will treat this as
try:
...
except:
# some exception occurred
typ = sys.exc_type
exc = sys.exc_value
if issubclass(typ, SomeError):
v = exc
...
else:
raise # propagate!
(where typ and exc are internal variables)
</F>
Rick Wotnaz
news:mailman.750.11321596...@python.org:
Thank you (and Roy Smith) for helping to clarify this. I see that
my mental image of an Exception (which, I admit, was not based on
extensive R'ing of TFM) was way off. Judging by Steven D'Aprano's
code sample, I'm not the only one who was mistaken about the nature
of v in your example. I'd always assumed it was the human-
readable string associated with the TypeError. Wrong, I see.
--
rzed
Steven D'Aprano
[cutting to the important bit]
>> except TypeError, msg:
>> if msg == "iteration over non-sequence":
>> # handle non-iterable case
> Does this in fact work on your system? On mine (2.4.1 (#65, Mar 30
> 2005, 09:13:57) [MSC v.1310 32 bit (Intel)]), it doesn't seem to.
Dammit, that will teach me not to test my code before posting.
No it doesn't: msg is an object of type exceptions.TypeError.
The easy fix is to just coerce it to a string:
if str(msg) == "iteration over non-sequence":
which *does* work on my system. But perhaps a better way is to do this:
# Create an instance of the exception you expect:
try:
for i in 0:
pass
except TypeError, ITER_OVER_NON_SEQ:
pass
# Now run your code...
try:
...blah blah blah...
except TypeError, msg
if str(msg) == str(ITER_OVER_NON_SEQ):
...blah blah blah...
This means we're no longer assuming what the error message will be,
which makes our code a lot more future-proof and implementation-proof: if
some version of Python changes the error string from "iteration over
non-sequence" to something else, the code should continue to work
correctly.
--
Steven.
Fredrik Lundh
> This means we're no longer assuming what the error message will be,
> which makes our code a lot more future-proof and implementation-proof: if
> some version of Python changes the error string from "iteration over
> non-sequence" to something else, the code should continue to work
> correctly.
Alex has already posted the right way to do this. can you please stop
</F>
Fredrik Lundh
posting crappy non-solutions to a problem that has a very simple solution (split
things up), that should be obvious to anyone who didn't sleep through exceptions
101.
</F>
Bengt Richter
>Rick Wotnaz wrote.
>
>> ... which leads me to belive that 'msg' is not type(str). It can be
>> coerced (str(msg).find works as expected). But what exactly is msg?
>> It appears to be of <type 'instance'>, and does not test equal to a
>> string.
>
>it's an instance of the exception type, of course.
>
>:::
>
>if you do
>
> raise SomeError, value
>
>Python will actually do
>
> raise SomeError(value)
Depending on what you mean by "actually" I guess ...
(I'm sure you know this and more ;-)
>>> dis.dis(compile('raise SomeError, value','','exec'))
1 0 LOAD_NAME 0 (SomeError)
3 LOAD_NAME 1 (value)
6 RAISE_VARARGS 2
9 LOAD_CONST 0 (None)
12 RETURN_VALUE
>>> dis.dis(compile('raise SomeError(value)','','exec'))
1 0 LOAD_NAME 0 (SomeError)
3 LOAD_NAME 1 (value)
6 CALL_FUNCTION 1
9 RAISE_VARARGS 1
12 LOAD_CONST 0 (None)
15 RETURN_VALUE
I guess that comes from the grammar of the raise statement, i.e.,
raise_stmt: 'raise' [test [',' test [',' test]]]
which allows up to three arguments for raise, apparently all general
expressions, but with some specific run-time requirements for what
combinations of argument expression values are allowable.
I.e., it seems (I haven't looked in ceval.c(?)) that RAISE_VARARGS must
look at the first item in its count of args on the stack, and in the
SomeError, value case find an exception class, and decide to instantiate it
and throw the instance, but if it finds the instance ready made, as in
SomeError(value), it must skip the instantiation (and disallow further args BTW).
>>> raise Exception, 'arg'
Traceback (most recent call last):
File "<stdin>", line 1, in ?
Exception: arg
>>> raise Exception('arg')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
Exception: arg
Those looks the same, but sensitivity to the type of the first arg is revealed by
>>> raise Exception('arg'), 'what now?'
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: instance exception may not have a separate value
>>> raise Exception('arg', 'what now?')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
Exception: ('arg', 'what now?')
>
>(that is, create a SomeError exception and pass the value as its
>first argument).
>
>you can use either form in your code (I prefer the latter myself).
>
Just to drive home the general expression allowability in raise,
and instance vs class as the first arg:
>>> extab = [StopIteration('stop stop ;-)'), ValueError('wrong value')]
>>> raise extab[1]
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: wrong value
>>> raise extab[0]
Traceback (most recent call last):
File "<stdin>", line 1, in ?
StopIteration: stop stop ;-)
>>> extab = [StopIteration, 'stop stop ;-)', ValueError, 'wrong value']
>>> raise extab[2], extab[3]
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: wrong value
>>> raise extab[0], extab[1]
Traceback (most recent call last):
File "<stdin>", line 1, in ?
StopIteration: stop stop ;-)
Ok, I'll stop ;-)
Regards,
Bengt Richter
Roy Smith
># Create an instance of the exception you expect:
>try:
> for i in 0:
> pass
>except TypeError, ITER_OVER_NON_SEQ:
> pass
># Now run your code...
>try:
> ...blah blah blah...
>except TypeError, msg
> if str(msg) == str(ITER_OVER_NON_SEQ):
> ...blah blah blah...
>
>This means we're no longer assuming what the error message will be,
>which makes our code a lot more future-proof and implementation-proof: if
>some version of Python changes the error string from "iteration over
>non-sequence" to something else, the code should continue to work
>correctly.
Now you're assuming that the message is a constant for all TypeErrors
caused by attempting to iterate over a non-iterable object. I don't
see anything in the docs which prevents
try:
for i in 0:
pass
except TypeError, ex:
print ex
from printing "Cannot iterate over integer with value 0", or even
"Cannot iteratate over object at address <0x456778>". Let's not even
talk about some future implementor who decides that the strings should
contain timestamps. None of these things are forbidden by the spec,
therefore you should assume they will all happen.
The best solution I've seen suggested so far is to coerce the
passed-in object to an iter and see what happens:
try:
thingieIterator = iter (thingie)
except TypeError:
raise NonIterableThingieError
Once you've done that, I think you can either do:
for item in thingie:
or
for item in thingieIterator:
with equal effect. And I'm reasonably sure this will work fine with
non-idempotent thingies.
Steven D'Aprano
Has he? Where and when? Is my crystal ball broken
again? I hate it when I miss the one Usenet post all
the cool kids are talking about.
> can you please stop
Can I please stop what? Using punctuation and
capitalization correctly? Learning? Thinking for
myself? Posting to Usenet? Programming in Python? Your
post is unclear.
--
Steven.
Steven D'Aprano
for so gently pointing me at this post. Just one
question, if I may:-
Alex Martelli wrote:
> It's not hard...:
>
> try:
> _it = iter(whatever)
> except TypeError:
> print 'non-iterable'
> else:
> for i in _it: # etc, etc
Alas and alack, I have to write code which is backwards
compatible with older versions of Python:
Python 2.1.1 (#1, Aug 25 2001, 04:19:08)
[GCC 3.0.1] on sunos5
Type "copyright", "credits" or "license" for more
information.
>>> iter
Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'iter' is not defined
What should I do when I can't rely on functions that
don't exist in older versions of Python?
--
Steven.
Steven D'Aprano
> Now you're assuming that the message is a constant for all TypeErrors
> caused by attempting to iterate over a non-iterable object.
Yes I am, and of course you are correct that that could
change and so shouldn't be relied on.
Seems that my crappy solution is not quite as
future-proof as I thought it was. Since a better
solution does exist, I agree it should be used in
preference to my crappy one, assuming you are running a
recent enough version of Python.
--
Steven.
Duncan Booth
> What should I do when I can't rely on functions that
> don't exist in older versions of Python?
>
Ideally:
if 'iter' not in dir(__builtins__):
import sys
sys.exit('Archaic Python not supported, please upgrade')
Alternatively fill in the blanks with appropriate fallback code:
if 'iter' not in dir(__builtins__):
def iter(obj):
... remainder of definition left as an exercise ...
__builtins__.iter = iter
where iter does roughly what the iter builtin does.
Roy Smith
> Alas and alack, I have to write code which is backwards
> compatible with older versions of Python:
> [...]
> NameError: name 'iter' is not defined
>
> What should I do when I can't rely on functions that
> don't exist in older versions of Python?
It really sucks trying to support obsolete environments. Been there, done
that. But sometimes you just have to pull out the duct tape and deal with
the problem any way you can. Parsing undocumented exception strings is
sort of like screen scraping; it's good to know that it's possible as an
absolute last resort, but you really want to put some effort into finding a
better way.
You could use two nested try blocks that both catch TypeErrors, one
enclosing the entire loop, the other enclosing just the body. Anything
that gets caught by the outer try had to have been generated within the
loop control code itself:
-------------------------
Roy-Smiths-Computer:play$ cat iter.py
#!/usr/bin/env python
def doSomething (obj):
sum = 0
try:
for i in obj:
try:
sum += i
except TypeError, ex:
print "exception (%s) in the inner block" % ex
return
except TypeError, ex:
print "non iterable object (%s)" % ex
return
print "sum = %d" % sum
doSomething ([1, 2, 3])
doSomething (["one", "two", "three"])
doSomething (0)
Roy-Smiths-Computer:play$ ./iter.py
sum = 6
exception (unsupported operand type(s) for +=: 'int' and 'str') in the
inner block
non iterable object (iteration over non-sequence)
-------------------------
This is pretty ugly, but I think it does what you want. It could be made a
bit less ugly by refactoring the body of the for loop into another function.
BTW, if you do resort to trying to parse the exception messages, and it
breaks in some future code, you might have me to blame. I submitted a bug
report a while back which, if it's ever acted on, would break your code!
http://sourceforge.net/tracker/index.php?func=detail&aid=1187437&group_id=54
70&atid=105470
Micah Elliott
> Steven D'Aprano wrote:
> > What should I do when I can't rely on functions that
> > don't exist in older versions of Python?
> sys.exit('Archaic Python not supported, please upgrade')
+1 QOTW.
I recently gave up on trying to support (backport to) pre-2.2 in my
projects. It's been ~3 years since 2.2 released and that seem like a
pretty reasonable support window. I wound up with HACK and
PORTABILITY comments all over the place and too much boilerplate. Now
I go with a startup check on sys.hexversion, and exit with similar
wording.
--
_ _ ___
|V|icah |- lliott <>< m...@micah.elliott.name
" " """
Roy Smith
> I recently gave up on trying to support (backport to) pre-2.2 in my
> projects. It's been ~3 years since 2.2 released and that seem like a
> pretty reasonable support window.
It depends on what you're doing. If you're developing for in-house
use, you have control over what you run and can upgrade whenever you
feel you can justify devoting the resources to the upgrade effort. If
you're a one-man band, upgrading to a new version may be an hour's
work to download and install the latest and greatest. If you're a cog
in a large organization, it may be quite a bit more hassle.
If you're developing for outside customers, it's a whole different
story. Lots of people are running operating systems which are several
years old (heck, I'm typing this on a Windows-2000 box), and those
operating systems may have shipped with versions of langauges which
were several years old at the time they shipped.
Consider the following conversation with a prospective customer:
Prospect: "I really love your product, the price is OK, and I've got
budget approval for the purchase. Now, what do we need to run it?"
You: "All you need is a box running Python 2.2".
Prospect: "Oh, bummer, we run 2.1 [or 2.0, or 1.5.2] in-house on all
our machines".
You: "No problem, it's easy to upgrade to 2.2"
Prospect: "Unfortunately, not around here it isn't. I.T. owns the
servers, and they won't do the upgrade. Looks like we won't be able
to use your product after all. Sorry."
I'm not saying you shouldn't use new stuff, but don't fool yourself
about how long a lifetime old versions have in the field. And, the
old adage that "The customer is always right" is a good one.
Steve Holden
> On Nov 17, Duncan Booth wrote:
>
>>
>>>What should I do when I can't rely on functions that
>>>don't exist in older versions of Python?
>
>
>
>
>
> I recently gave up on trying to support (backport to) pre-2.2 in my
> projects. It's been ~3 years since 2.2 released and that seem like a
> PORTABILITY comments all over the place and too much boilerplate. Now
> I go with a startup check on sys.hexversion, and exit with similar
> wording.
>
releases products that run on everything back to Python 1.5.2. Kudos to
the effbot!
regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/
Fredrik Lundh
> Alas and alack, I have to write code which is backwards
> compatible with older versions of Python:
>
> Python 2.1.1 (#1, Aug 25 2001, 04:19:08)
> [GCC 3.0.1] on sunos5
> Type "copyright", "credits" or "license" for more
> information.
> >>> iter
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> NameError: name 'iter' is not defined
>
> What should I do when I can't rely on functions that
> don't exist in older versions of Python?
python 2.1 doesn't support iterators, so that question doesn't
make much sense.
$ python2.1
>>> class iterable:
... def __iter__(self):
... print "ITER"
... return self
... def next(self):
... print "NEXT"
... return 1
...
>>> for i in iterable():
... print i
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: iterable instance has no attribute '__getitem__'
if you want to write code that runs under 2.1, you have to write
your code in terms of what 2.1 supports. Python's compatibility
model means that code written for old versions still work in new
versions; it doesn't mean that code written for new versions will
always work properly (or raise proper exceptions) in old versions.
</F>
Steven D'Aprano
> Steven D'Aprano wrote:
>
>
>>Alas and alack, I have to write code which is backwards
>>compatible with older versions of Python:
[snip]
>>What should I do when I can't rely on functions that
>>don't exist in older versions of Python?
>
>
> python 2.1 doesn't support iterators, so that question doesn't
> make much sense.
The _question_ doesn't make much sense? I could
understand you saying that backwards-compatibility is
"not important [to me]" but to say that the very
question of how to maintain backwards compatibility
makes little sense is a bit extreme, don't you think?
Fredrik, I bow to your superior knowledge about Python,
no sarcasm intended, and I've learnt a lot from your
posts, thank you. But this is not one of your shining
moments. Your attitude was utterly dismissive: the
"right way" to solve the problem of recognising
iterables was to use iter, and that's all that needs to
be said.
The problem of how to recognise iterables did not come
into existence with version 2.2, and backwards
compatibility is sometimes a real requirement. A few
months back I had to mangle some Py2.4 code so that it
would run under version 2.0, and wasn't that fun.
> if you want to write code that runs under 2.1, you have to write
> your code in terms of what 2.1 supports.
Do you think I don't know this?
I never imagined for an instant that Python 2.1 would
somehow magically be able to use features that didn't
exist in Python 2.1. But then it wasn't me saying that
there was nothing to discuss, the "right way" is to use
iter(), end of story.
If I have to write code that can't rely on iter()
existing in the language, what should I do?
Are there practical idioms for solving the metaproblem
"solve problem X using the latest features where
available, otherwise fall back on older, less powerful
features"?
For instance, perhaps I might do this:
try:
built_in_feature
except NameError:
# fall back on a work-around
from backwards_compatibility import \
feature as built_in_feature
Do people do this or is it a bad idea?
Are there other techniques to use? Obviously refusing
to run is a solution (for some meaning of "solution"),
it may even be a practical solution for some cases, but
is it the only one?
In the specific case of iter(), are there good
alternative ways of detecting an iterable without
consuming it?
--
Steven.
Alex Martelli
> In the specific case of iter(), are there good
> alternative ways of detecting an iterable without
> consuming it?
Not a problem I've had often, but when I did, if I recall correctly, I
did something like:
try:
iter
except NameError:
def isiterable(x):
try: x[0]
except Exception: return 0
else: return 1
else:
def isiterable(x):
try: iter(x)
except TypeError: return 0
else: return 1
Not True/False because they wouldn't be available in 2.0 any more than
iter would. "Consuming" didn't really come into consideration for the
backwards compatibility part because only objects indexable with
integers, 0 and up (and raising IndexError at some point) were usable in
for statements in old Pythons, there was no "consuming". The tests here
are not 100% reliable, of course -- in 2.0, a dict which just happens to
have a 0 key would erroneously pass; but short of actually trying the
loop, there's no infallible way that I know of.
One possibility (that I haven't tried in real life) to deal with the
hellish situation of having to write code that's able to loop on an
iterable but fail softly otherwise, while not hiding errors in the
loop's body, would be the suitably hellish...:
class NotATypeError(Exception):
def __init__(self, e, t):
self.e = e
self.t = t
try:
try:
for item in whatever:
try:
...loop body here...
except TypeError, e, t:
raise NotATypeError(e, t)
except TypeError:
....fail-softly code...
except NotATypeError, x:
raise TypeError, x.e, x.t
This kind of spaghetti code is what gives backwards compatibility its
bad name, of course. Be sure that you're getting paid for this in
proportion to its ugliness, and your finances should be set for life.
Alex
Steven D'Aprano
> "Consuming" didn't really come into consideration for the
> backwards compatibility part because only objects indexable with
> integers, 0 and up (and raising IndexError at some point) were usable in
> for statements in old Pythons, there was no "consuming".
Ah yes, of course. How quickly we forget: you couldn't
say "for line in file...". No generators either.
But there was still at least one object that was
consumable: xreadlines.
Still, I think using an xreadlines object is an unusual
enough case that I'm not going to lose any sleep about
it. Document it as a known issue and forget it *wink*
[snip]
> This kind of spaghetti code is what gives backwards compatibility its
> bad name, of course. Be sure that you're getting paid for this in
> proportion to its ugliness, and your finances should be set for life.
Hah, I wish!
Thanks for the assistance, I learnt a lot. Let's hope I
don't have to use it ever again...
--
Steven.
Fredrik Lundh
> > In the specific case of iter(), are there good
> > alternative ways of detecting an iterable without
> > consuming it?
>
> Not a problem I've had often, but when I did, if I recall correctly, I
> did something like:
>
> try:
> iter
> except NameError:
> def isiterable(x):
> try: x[0]
> except Exception: return 0
> else: return 1
> else:
> def isiterable(x):
> try: iter(x)
> except TypeError: return 0
> else: return 1
>
> Not True/False because they wouldn't be available in 2.0 any more than
> iter would. "Consuming" didn't really come into consideration for the
> backwards compatibility part because only objects indexable with
> integers, 0 and up (and raising IndexError at some point) were usable in
> for statements in old Pythons, there was no "consuming".
Before the iterator protocol was added, using the sequence protocol
to implement "forward-only iterators" weren't uncommon. e.g.
class my_file_iterator:
def __init__(self, file):
self.file = file
def __getitem__(self, index):
line = self.file.readline()
if not line:
raise IndexError
return line
(for extra points, add code that makes sure that the index matches
the line number)
To test for this, you could look for __getitem__ methods on instance
objects, and use operator.isSequenceType on everything else, but that
doesn't distinguish between sequences and mappings.
> short of actually trying the loop, there's no infallible way that I know of.
Exactly. And iter() can fail too, even if it's far less likely that you
stumble upon an iterable that misbehaves if you call __iter__ one
extra time. Which is why the portable pragmatic pythonic solution
is to design your program so it doesn't depend on type/interface
testing.
If you want to loop over things, loop over things, and leave it to
Python to raise an exception if you get something that doesn't
support looping.
And unless you have very good reasons, you should treat that
exception as a bug in your program.
</F>
Tom Anderson
> Are there practical idioms for solving the metaproblem "solve problem X
> using the latest features where available, otherwise fall back on older,
> less powerful features"?
>
> For instance, perhaps I might do this:
>
> try:
> built_in_feature
> except NameError:
> # fall back on a work-around
> from backwards_compatibility import \
> feature as built_in_feature
>
> Do people do this or is it a bad idea?
From some code i wrote yesterday, which has to run under 2.2:
try:
True
except NameError:
True = 1 == 1
False = 1 == 0
Great minds think alike!
As for whether it's a bad idea, well, bad or not, it certainly seems like
the least worst.
> Are there other techniques to use? Obviously refusing to run is a
> solution (for some meaning of "solution"), it may even be a practical
> solution for some cases, but is it the only one?
How about detecting which environment you're in, then running one of two
entirely different sets of code? Rather than trying to construct modern
features in the antique environment, write code for each, using the local
idioms. The trouble with this is that you end up with massive duplication;
you can try to factor out the common parts, but i suspect that the
differing parts will be a very large fraction of the codebase.
> If I have to write code that can't rely on iter() existing in the
> language, what should I do?
Can you implement your own iter()? I have no idea what python 2.0 was
like, but would something like this work:
class _iterator:
def __init__(self, x):
self.x = x
self.j = 0
def next(self):
self.j = self.j + 1
return self.x.next()
def __getitem__(self, i):
if (i != self.j):
raise ValueError, "out of order iteration"
try:
return self.next()
except StopIteration:
raise IndexError
def __iter__(self):
return self
# hopefully, we don't need this, but if we do ...
def __len__(self):
return sys.maxint # and rely on StopIteration to stop the loop
class _listiterator(_iterator):
def next(self):
try:
item = self.x[self.j]
self.j = self.j + 1
return item
except IndexError:
raise StopIteration
def __getitem__(self, i):
if (i != self.j):
raise ValueError, "out of order iteration"
self.j = self.j + 1
return self.x[i]
import types
def iter(x):
# if there's no hasattr, use explicit access and try-except blocks
# handle iterators and iterables from the future
if hasattr(x, "__iter__"):
return _iterator(x.__iter__())
# if there's no __getitem__ on lists, try x[0] and catch the exception
# but leave the __getitem__ test to catch objects from the future
if hasattr(x, "__getitem__"):
return _listiterator(x)
if type(x) == types.FileType:
return _fileiterator(x) # you can imagine the implementation of this
# insert more tests for specific types here as you like
raise TypeError, "iteration over non-sequence"
?
NB haven't actually tried to run that code.
tom
--
I'm angry, but not Milk and Cheese angry. -- Mike Froggatt
Fredrik Lundh
> How about detecting which environment you're in, then running one of two
> entirely different sets of code? Rather than trying to construct modern
> features in the antique environment, write code for each, using the local
> idioms. The trouble with this is that you end up with massive duplication;
> you can try to factor out the common parts, but i suspect that the
> differing parts will be a very large fraction of the codebase.
That sounds like the worst possible way of writing portable code (unless you
have huge amounts of time and money, and wasting them don't really matter
to you).
In my experience, a more practical approach is to write code that targets the
2.0/2.1 platform, and selectively (based on feature tests, not version numbers)
replace portions with more efficient mechanisms from newer versions. (most
of those are found in the standard library, not in the language itself).
(this is related to another popular myth: that code that uses "the old way" is
automatically slower than code that uses newfangled stuff, also when running
on recent Python versions. that's very seldom true; new Python versions runs
"old" code faster too.)
If you want to support 1.5.2 or Unicode-less Python builds, you need to add
some fallbacks too, but that's mostly trivial. The easiest way is to pass 8-bit
strings through "as is", leaving encoding issues to the application.
>> If I have to write code that can't rely on iter() existing in the
>> language, what should I do?
>
> Can you implement your own iter()? I have no idea what python 2.0 was
> like, but would something like this work:
Python 2.0 had sequences. Python 2.4 has sequences. Iterators don't really
add anything. Emulating them in older versions is mostly pointless.
I've said it many times, and I'll say it again: the only fundamentally new concept
that has been added since Python 1.5.2 is generators. And a lot of the stuff you
want generators for can be emulated with sequences and extra buffer layers.
All the rest is just coloured frosting; you can save a line here and there (which is
a real benefit only if your return key is broken), but that's about it.
</F>
Roy Smith
> I've said it many times, and I'll say it again: the only fundamentally
> new concept that has been added since Python 1.5.2 is generators.
> [...]
> All the rest is just coloured frosting
In my mind, the biggest thing since 1.5.2 is string methods. They are
perhaps, as you say, just frosting, but they're very good tasting frosting
:-)