Pythonic way for missing dict keys
Alex Popescu
I am pretty sure this has been asked a couple of times, but I don't seem
to find it on the archives (Google seems to have a couple of problems
lately).
I am wondering what is the most pythonic way of dealing with missing
keys and default values.
According to my readings one can take the following approaches:
1/ check before (this has a specific name and acronym that I haven't
learnt yet by heart)
if not my_dict.has_key(key):
my_obj = myobject()
my_dict[key] = my_obj
else:
my_obj = my_dict[key]
2/ try and react on error (this has also a specific name, but...)
try:
my_obj = my_dict[key]
except AttributeError:
my_obj = myobject()
my_dict[key] = my_obj
3/ dict.get usage:
my_obj = my_dict.get(key, myobject())
I am wondering which one is the most recommended way? get usage seems
the clearest, but the only problem I see is that I think myobject() is
evaluated at call time, and so if the initialization is expensive you
will probably see surprises.
thanks in advance,
./alex
--
.w( the_mindstorm )p.
Neil Cerutti
> Hi all!
>
> I am pretty sure this has been asked a couple of times, but I
> don't seem to find it on the archives (Google seems to have a
> couple of problems lately).
>
> I am wondering what is the most pythonic way of dealing with missing
> keys and default values.
>
> According to my readings one can take the following approaches:
There's also the popular collections.defaultdict.
Usually, the get method of normal dicts is what I want. I use a
defaultdict only when the implicit addition to the dictionary of
defaulted elements is what I really want.
--
Neil Cerutti
Carsten Haese
> Hi all!
>
> I am pretty sure this has been asked a couple of times, but I don't seem
> to find it on the archives (Google seems to have a couple of problems
> lately).
>
> I am wondering what is the most pythonic way of dealing with missing
> keys and default values.
>
> According to my readings one can take the following approaches:
>
> learnt yet by heart)
>
> if not my_dict.has_key(key):
> my_obj = myobject()
> my_dict[key] = my_obj
> else:
> my_obj = my_dict[key]
This is called "Look before you leap." Note that "if key in my_dict" is
preferred over has_key().
> 2/ try and react on error (this has also a specific name, but...)
>
> try:
> my_obj = my_dict[key]
> except AttributeError:
> my_obj = myobject()
> my_dict[key] = my_obj
>
This is called "It's easier to ask for forgiveness than permission."
> 3/ dict.get usage:
>
> my_obj = my_dict.get(key, myobject())
>
> I am wondering which one is the most recommended way? get usage seems
> the clearest, but the only problem I see is that I think myobject() is
> evaluated at call time, and so if the initialization is expensive you
> will probably see surprises.
All ways are equally acceptable. The main difference is in the amount of
optimism the code conveys about the existence of the key. Number 1 and 3
convey the notion that you expect the key to be missing, that it's
normal for the key to be missing. Number 2 conveys the notion that the
key is only missing under exceptional circumstances.
As you noted, number 3 is not good if it's expensive to initialize a
myobject() instance, because the object will be created regardless of
whether it'll be needed. In that case, you should go with number 1, or
the following variant of number 3:
my_obj = my_dict.get(key, None)
if my_obj is None: my_obj = myobject()
This only works if the dictionary doesn't contain Nones already. If it
does, you would create a cheap sentinel object instead:
sentinel = object()
my_obj = my_dict.get(key, sentinel)
if my_obj is sentinel: my_obj = myobject()
HTH,
--
Carsten Haese
http://informixdb.sourceforge.net
Alex Popescu
news:slrnfa2325...@FIAD06.norwich.edu:
This looks like the closest to my needs, but in my case the default value
involves the creation of a custom object instance that is taking parameters
from the current execution context, so I am not very sure I can use it.
Jakub Stolarski
add value to dict.
As it was mentioned before, use:
1 - if you expect that there's no key in dict
2 - if you expect that there is key in dict
Alex Popescu
news:1184961448.4...@k79g2000hse.googlegroups.com:
I may be missing something but I think the 3 approaches are completely
equivalent in terms of functionality.
Carsten Haese
> Carsten Haese <car...@uniqsys.com> wrote in
> news:1184959878.3...@dot.uniqsys.com:
> > On Fri, 2007-07-20 at 19:08 +0000, Alex Popescu wrote:
> >> Hi all!
> >>
>
> >
> >
> >
> > This is called "Look before you leap." Note that "if key in my_dict"
> > is preferred over has_key().
> >
>
> readability/performance?
Yes, it's about both readability and performance:
[carsten@dot ~]$ python -m timeit -s "a={}" "a.has_key('foo')"
1000000 loops, best of 3: 0.601 usec per loop
[carsten@dot ~]$ python -m timeit -s "a={}" "'foo' in a"
1000000 loops, best of 3: 0.307 usec per loop
> Unfortunately, the get(key, None) suggestion is not 100% equivalent to
> my posted code, and once I add the addition of the new object to the map
> I think 1/ and 3/ are equivalent (if no other
> readability/performance/etc is involved).
Indeed, I missed that in 1/ and 2/ you're also adding the value to the
dictionary. You could use setdefault instead of get:
my_obj = my_dict.setdefault(key, myobject())
But if you want to defer creation of the default value until it's
actually needed, you're better off with the "Look before you leap"
approach.
Carsten Haese
> Neil Cerutti <hor...@yahoo.com> wrote in
> news:slrnfa2325...@FIAD06.norwich.edu:
>
> involves the creation of a custom object instance that is taking parameters
> from the current execution context, so I am not very sure I can use it.
If by "current execution context" you mean globally visible names, this
should still be possible:
>>> from collections import defaultdict
>>> def make_default():
... return x+y
...
>>> dd = defaultdict(make_default)
>>> x = 40
>>> y = 2
>>> print dd[0]
42
>>> x = "Dead"
>>> y = " Parrot"
>>> print dd[1]
Dead Parrot
>>> print dd
defaultdict(<function make_default at 0xb7f71e2c>, {0: 42, 1: 'Dead Parrot'})
HTH,
Steven D'Aprano
> I am wondering what is the most pythonic way of dealing with missing
> keys and default values.
[snip three versions]
Others have already mentioned the collections.defaultdict type, however it
seems people have forgotten about the setdefault method of dictionaries.
value = somedict.setdefault(key, defaultvalue)
The disadvantage of setdefault is that the defaultvalue has to be created
up front. The disadvantage of collections.defaultdict is that the "default
factory" function takes no arguments, which makes it rather less than
convenient. One can work around this using global variables:
# The default value is expensive to calculate, and known
# only at runtime.
>>> expensivefunction = lambda x: str(x)
>>> D = collections.defaultdict(lambda : expensivefunction(context))
# lots of code goes here...
# set context just before fetching from the default dict
>>> context = 42
>>> value = D['key']
>>> print value, D
42 defaultdict(<function <lambda> at 0xb7eb4fb4>, {'key': '42'})
but one should be very leery of relying on global variables like that.
That suggests the best solution is something like this:
def getdefault(adict, key, expensivefunction, context):
if key in adict:
return adict[key]
else:
value = expensivefunction(context)
adict[key] = value
return value
--
Steven.
Rustom Mody
>>> x in dic
is cheaper than
>>> dic.has_key(x)
??
Carsten Haese
I won't claim to know Python internals, but compiling and disassembling the
expressions in question reveals the reason:
>>> from compiler import compile
>>> from dis import dis
>>> dis(compile("dic.has_key(x)","","eval"))
1 0 LOAD_NAME 0 (dic)
3 LOAD_ATTR 1 (has_key)
6 LOAD_NAME 2 (x)
9 CALL_FUNCTION 1
12 RETURN_VALUE
>>> dis(compile("x in dic","","eval"))
1 0 LOAD_NAME 0 (x)
3 LOAD_NAME 1 (dic)
6 COMPARE_OP 6 (in)
9 RETURN_VALUE
"dic.has_key(x)" goes through an attribute lookup to find the function that
looks for the key. "x in dic" finds the function more directly.
Bruno Desthuilliers
> Hi all!
>
> I am pretty sure this has been asked a couple of times, but I don't seem
> to find it on the archives (Google seems to have a couple of problems
> lately).
>
> I am wondering what is the most pythonic way of dealing with missing
> keys and default values.
>
> According to my readings one can take the following approaches:
>
> 1/ check before (this has a specific name and acronym that I haven't
> learnt yet by heart)
>
> if not my_dict.has_key(key):
> my_obj = myobject()
> my_dict[key] = my_obj
> else:
> my_obj = my_dict[key]
if key not in my_dict:
my_obj = my_dict[key] = myobject()
else:
my_obj = my_dict[key]
> 2/ try and react on error (this has also a specific name, but...)
>
> try:
> my_obj = my_dict[key]
> except AttributeError:
> my_obj = myobject()
> my_dict[key] = my_obj
cf above for a shortcut...
> 3/ dict.get usage:
>
> my_obj = my_dict.get(key, myobject())
Note that this last one won't have the same result, since it won't store
my_obj under my_dict[key]. You'd have to use dict.setdefault :
my_obj = my_dict.setdefault(key, myobject())
> I am wondering which one is the most recommended way?
It depends on the context. wrt/ 1 and 2, use 1 if you expect that most
of the time, my_dict[key] will not be set, and 2 if you expect that most
of the time, my_dict[key] will be set.
> get usage seems
> the clearest, but the only problem I see is that I think myobject() is
> evaluated at call time,
Myobject will be instanciated each time, yes.
> and so if the initialization is expensive you
> will probably see surprises.
No "surprise" here, but it can indeed be suboptimal if instanciating
myobject is costly.
Bruno Desthuilliers
> Jakub Stolarski <jakub.s...@gmail.com> wrote in
> news:1184961448.4...@k79g2000hse.googlegroups.com:
>
>
>>Version 1 and 2 do different thing than version 3. The latter doesn't
>>add value to dict.
>>
>>As it was mentioned before, use:
>>1 - if you expect that there's no key in dict
>>2 - if you expect that there is key in dict
>>
>
>
> I may be missing something
You are.
> but I think the 3 approaches are completely
> equivalent in terms of functionality.
d = dict()
answer = d.get('answer', 42)
answer in d
=> False
Alex Martelli
Yup, it's mostly that, as microbenchmarking can confirm:
brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' 'f(23)'
10000000 loops, best of 3: 0.146 usec per loop
brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' '23 in d'
10000000 loops, best of 3: 0.142 usec per loop
brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' 'f(23)'
10000000 loops, best of 3: 0.146 usec per loop
brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' '23 in d'
10000000 loops, best of 3: 0.142 usec per loop
brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' 'd.has_key(23)'
1000000 loops, best of 3: 0.278 usec per loop
brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' 'd.has_key(23)'
1000000 loops, best of 3: 0.275 usec per loop
the in operator still appears to have a tiny repeatable advantage (about
4 nanoseconds on my laptop) wrt even the hoisted method, but the
non-hoisted method, due to repeated lookup, is almost twice as slow
(over 100 nanoseconds penalty, on my laptop).
Alex
Alex Popescu
news:46a20fda$0$27858$426a...@news.free.fr:
> Alex Popescu a écrit :
>> Jakub Stolarski <jakub.s...@gmail.com> wrote in
>
>
> [snip...]
>
>
> d = dict()
> answer = d.get('answer', 42)
> answer in d
> => False
>
Thanks. I think to make the 3rd approach completely equivalent I should
have been using d.setdefault(key, myojbect()).
Duncan Booth
Some special methods are optimised by having a reserved slot in the data
structure used to implement a class. The 'in' operator uses one of these
slots so it can bypass all the overheads of looking up an attribute such as
'has_key'.
Zentrader
> "Rustom Mody" <rustompm...@gmail.com> wrote:
> > Can someone who knows about python internals throw some light on why
> >>>> x in dic
> > is cheaper than
> >>>> dic.has_key(x)
>
> > ??
>
>From the 2.6 PEP #361 (looks like dict.has_key is deprecated)
Python 3.0 compatability: ['compatibility'-->someone should use a
spell-checker for 'official' releases]
- warnings were added for the following builtins which no
longer exist in 3.0:
apply, callable, coerce, dict.has_key, execfile, reduce,
reload
genro
<bdesth.quelquech...@free.quelquepart.fr> wrote:
>
> Myobject will be instanciated each time, yes.
>
> > and so if the initialization is expensive you
> > will probably see surprises.
>
> No "surprise" here, but it can indeed be suboptimal if instanciating
> myobject is costly.
What about this way ?
my_obj = my_dict.get(key) or my_dict.setdefault(key,myobject())
Ciao
G.
Marc 'BlackJack' Rintsch
Reduces the unnecessary instantiation of `myobject` to "false" objects.
May be not good enough.
Ciao,
Marc 'BlackJack' Rintsch
Alex Popescu
@x40g2000prg.googlegroups.com:
> On Jul 21, 7:48 am, Duncan Booth <duncan.bo...@invalid.invalid> wrote:
>
> [snip...]
>
>
>>From the 2.6 PEP #361 (looks like dict.has_key is deprecated)
> Python 3.0 compatability: ['compatibility'-->someone should use a
> spell-checker for 'official' releases]
> - warnings were added for the following builtins which no
> longer exist in 3.0:
> apply, callable, coerce, dict.has_key, execfile, reduce,
> reload
>
I see... what that document doesn't describe is the alternatives to be
used. And I see in that list a couple of functions that are probably used a
lot nowadays (callable, reduce, etc.).
bests,
./alex
--
.w the_mindstorm )p.
John J. Lee
callable and reduce are rarely used, at least in code I've seen. I
would agree there will be a large number of programs that contain one
or two calls to these functions, though. Certainly has_key will be
the most common of those listed above (but trivial to fix). apply
will be common in old code from the time of Python 1.5.2. execfile is
perhaps more common that callable (?) but again is really a "maybe 1
call in a big program" sort of thing. Anybody using coerce or reload
deserves to lose ;-)
John
Alex Popescu
> Alex Popescu <nospam.th...@gmail.com> writes:
>
>> Zentrader <zentr...@gmail.com> wrote in
>> news:1185041243.323915.161230 @x40g2000prg.googlegroups.com:
>>
>>> On Jul 21, 7:48 am, Duncan Booth <duncan.bo...@invalid.invalid>
>>> wrote:
>>>
>>> [snip...]
>>>
>>>
>>>>From the 2.6 PEP #361 (looks like dict.has_key is deprecated)
>>> Python 3.0 compatability: ['compatibility'-->someone should use a
>>> spell-checker for 'official' releases]
>>> - warnings were added for the following builtins which no
>>> longer exist in 3.0:
>>> apply, callable, coerce, dict.has_key, execfile,
>>> reduce,
>>> reload
>>>
>>
>> I see... what that document doesn't describe is the alternatives to
>> be used. And I see in that list a couple of functions that are
>> probably used a lot nowadays (callable, reduce, etc.).
>
> callable and reduce are rarely used, at least in code I've seen.
I thought G would be using that function a lot. Also, what is the
replacement of reduce? I think I remember seeing somewhere that lists
comprehension would be (but also remember the advise that reduce will be
quicker).
> Certainly has_key will be
> the most common of those listed above (but trivial to fix).
dict.has_key(key) becomes key in dict (correct?)
> apply
> will be common in old code from the time of Python 1.5.2.
I think there were some advises to not use apply.
> execfile is
> perhaps more common that callable (?) but again is really a "maybe 1
> call in a big program" sort of thing.
What is the replacement for this one?
tia,
Steve Holden
>>
>>> Zentrader <zentr...@gmail.com> wrote in
>>> news:1185041243.323915.161230 @x40g2000prg.googlegroups.com:
>>>
>>>> On Jul 21, 7:48 am, Duncan Booth <duncan.bo...@invalid.invalid>
>>>> wrote:
>>>>
>>>> [snip...]
>>>>
>>>>
>>>> >From the 2.6 PEP #361 (looks like dict.has_key is deprecated)
>>>> Python 3.0 compatability: ['compatibility'-->someone should use a
>>>> spell-checker for 'official' releases]
>>>> - warnings were added for the following builtins which no
>>>> longer exist in 3.0:
>>>> apply, callable, coerce, dict.has_key, execfile,
>>>> reduce,
>>>> reload
>>>>
>>> I see... what that document doesn't describe is the alternatives to
>>> be used. And I see in that list a couple of functions that are
>>> probably used a lot nowadays (callable, reduce, etc.).
>> callable and reduce are rarely used, at least in code I've seen.
>
> replacement of reduce? I think I remember seeing somewhere that lists
> comprehension would be (but also remember the advise that reduce will be
> quicker).
>
FAQ. He suggests that uses if reduce are almost always more readable
when written using a for loop and an accumulator variable.
http://www.artima.com/weblogs/viewpost.jsp?thread=211200
>> Certainly has_key will be
>> the most common of those listed above (but trivial to fix).
>
> dict.has_key(key) becomes key in dict (correct?)
>
>> apply
>> will be common in old code from the time of Python 1.5.2.
>
> I think there were some advises to not use apply.
>
Yup. Anyway there's a trivial translation for uses of apply.
apply(f, *args, **kw) => f(*args, **kw)
The converter will almost certainly do this for you.
>> execfile is
>> perhaps more common that callable (?) but again is really a "maybe 1
>> call in a big program" sort of thing.
>
I don't ever remember using execfile. It has always been a blot on the
landscape.
> What is the replacement for this one?
>
I'd have to guess, and will refrain from doing so.
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------
Steven D'Aprano
> j...@pobox.com (John J. Lee) wrote in news:87lkd0e...@pobox.com:
>
>> Alex Popescu <nospam.th...@gmail.com> writes:
>>
>>> Zentrader <zentr...@gmail.com> wrote in
>>> news:1185041243.323915.161230 @x40g2000prg.googlegroups.com:
>>>
>>>> On Jul 21, 7:48 am, Duncan Booth <duncan.bo...@invalid.invalid>
>>>> wrote:
>>>>
>>>> [snip...]
>>>>
>>>>
>>>>>From the 2.6 PEP #361 (looks like dict.has_key is deprecated)
>>>> Python 3.0 compatability: ['compatibility'-->someone should use a
>>>> spell-checker for 'official' releases]
>>>> - warnings were added for the following builtins which no
>>>> longer exist in 3.0:
>>>> apply, callable, coerce, dict.has_key, execfile,
>>>> reduce,
>>>> reload
>>>>
>>>
>>> I see... what that document doesn't describe is the alternatives to
>>> be used. And I see in that list a couple of functions that are
>>> probably used a lot nowadays (callable, reduce, etc.).
>>
>> callable and reduce are rarely used, at least in code I've seen.
>
> I thought G would be using that function a lot.
Who or what is G? Guido?
And which of the two functions mentioned is "that function"? callable()?
Instead of doing:
if callable(function): function()
you should do:
try:
function()
except TypeError:
pass
That should work for most uses of callable(), but isn't quite the same.
(What if function() has side-effects, or is expensive, and you want to
determine if it is callable, but not actually call it _now_?)
> Also, what is the replacement of reduce? I think I remember seeing
> somewhere that lists comprehension would be (but also remember the
> advise that reduce will be quicker).
No, a list comprehension isn't equivalent to reduce(). There is no
replacement for reduce().
Guido has essentially done the functional equivalent of deciding that
because he personally doesn't like for loops, they can and should be
written out as:
counter = 0
while counter < len(sequence):
item = sequence[counter]
do_something_with_item
counter += 1
because it is "easier to read" than a for loop.
And yes, if you think my example is extreme because nobody would be that
stupid, that's the point: removing reduce() _is_ that stupid (and it is
not often I say that about a language design decision by Guido!). Python
can, and probably will, get away with it only because reducing an iterable
to a single value is an uncommon operation, and the most common uses of it
(summing a sequence, finding the minimum and maximum) already have
workable replacements.
It's especially stupid because map() and filter(), both of which do have
simple, easy to understand, completely functional replacements, are going
to stay. reduce() doesn't, and it is going. That's nuts.
It's a shame really. Oh well, maybe it will sneak back in via a functional
module, or itertools, or something. What a waste, what a waste.
>> Certainly has_key will be
>> the most common of those listed above (but trivial to fix).
>
> dict.has_key(key) becomes key in dict (correct?)
Yes.
>> apply
>> will be common in old code from the time of Python 1.5.2.
>
> I think there were some advises to not use apply.
help(apply) already tells you how to replace apply():
Deprecated since release 2.3. Instead, use the extended call syntax:
function(*args, **keywords).
>> execfile is
>> perhaps more common that callable (?) but again is really a "maybe 1
>> call in a big program" sort of thing.
>
> What is the replacement for this one?
Slurp the file into a string, execute the string.
But the thing that makes my brain melt is reload(). No, reload() probably
isn't good enough to use in production code, but for interactive use it is
Good Enough. I'm boggled that it is going away!
--
Steven.
Duncan Booth
>
> Instead of doing:
>
>
> if callable(function): function()
>
> you should do:
>
> try:
> function()
> except TypeError:
> pass
>
>
> That should work for most uses of callable(), but isn't quite the
> same. (What if function() has side-effects, or is expensive, and you
> want to determine if it is callable, but not actually call it _now_?)
The replacement for callable(x) is simply hasattr(x, '__call__').
Once upon a time it may have been that functions didn't have a __call__
attribute (I haven't checked back on really old Pythons top see if this
was the case), but these
>> Also, what is the replacement of reduce? I think I remember seeing
>> somewhere that lists comprehension would be (but also remember the
>> advise that reduce will be quicker).
>
> No, a list comprehension isn't equivalent to reduce(). There is no
> replacement for reduce().
<snip>
There are of course several replacements for specialised uses of reduce:
sum, any, all. There is no general purpose replacement, but you can
write one in a few lines if you really need it.
>
> It's a shame really. Oh well, maybe it will sneak back in via a
> functional module, or itertools, or something. What a waste, what a
> waste.
I'm sure it will reappear in some other module, but that's the correct
place for a little used function, not in builtins.
Steven D'Aprano
> Steven D'Aprano <st...@REMOVE.THIS.cybersource.com.au> wrote:
>
>>
>> Instead of doing:
>>
>>
>> if callable(function): function()
>>
>> you should do:
>>
>> try:
>> function()
>> except TypeError:
>> pass
>>
>>
>> That should work for most uses of callable(), but isn't quite the
>> same. (What if function() has side-effects, or is expensive, and you
>> want to determine if it is callable, but not actually call it _now_?)
>
> The replacement for callable(x) is simply hasattr(x, '__call__').
> Once upon a time it may have been that functions didn't have a __call__
> attribute (I haven't checked back on really old Pythons top see if this
> was the case), but these
Your sentence seems to have run out early :)
Thanks, I didn't know that -- I remember that Back In The Day checking if
something was callable was tricky, because it could be a function, a
method, or a class with __call__. It makes much more sense to make all
callables go through __call__.
>>> Also, what is the replacement of reduce? I think I remember seeing
>>> somewhere that lists comprehension would be (but also remember the
>>> advise that reduce will be quicker).
>>
>> No, a list comprehension isn't equivalent to reduce(). There is no
>> replacement for reduce().
> <snip>
> There are of course several replacements for specialised uses of reduce:
> sum, any, all. There is no general purpose replacement, but you can
> write one in a few lines if you really need it.
True. But writing the same basic algorithm over and over again is about as
far from best practice as you can get without being a PHP programmer.
*wink* Currying is easy to do too, but Python has grown a function
functools.partial() to do it properly. That's as it should be.
(Yes, I know that's a misuse of the term currying, but it's a common one,
and easier than saying "Making a partial function".)
>> It's a shame really. Oh well, maybe it will sneak back in via a
>> functional module, or itertools, or something. What a waste, what a
>> waste.
>
> I'm sure it will reappear in some other module, but that's the correct
> place for a little used function, not in builtins.
Oh, I would have no problem at all with reduce being banished to the
functtools module. But it's a shame to have to go through the whole PEP
process, and risk Guido saying no.
--
Steven.
John J. Lee
[...]
> Yup. Anyway there's a trivial translation for uses of apply.
>
> apply(f, *args, **kw) => f(*args, **kw)
Steve means:
apply(f, args, kw) => f(*args, **kw)
John
Chris Mellon
It's *really* not the same. Not only does it call the function, which
has all kinds of potential problems like you listed, but this also
will convert TypeErrors in the called function into "this is not a
callable", which is almost certainly not what you want. hasattr(x,
'__call__') works but is annoying to write and hides intent. I'd be
happy if inspect grew iscallable.
Bruno Desthuilliers
> Alex Popescu <nospam.th...@gmail.com> writes:
>
>
>>Zentrader <zentr...@gmail.com> wrote in news:1185041243.323915.161230
>>@x40g2000prg.googlegroups.com:
>>
>>
>>>On Jul 21, 7:48 am, Duncan Booth <duncan.bo...@invalid.invalid> wrote:
>>>
>>>[snip...]
>>>
>>>
>>>>From the 2.6 PEP #361 (looks like dict.has_key is deprecated)
>>>Python 3.0 compatability: ['compatibility'-->someone should use a
>>>spell-checker for 'official' releases]
>>> - warnings were added for the following builtins which no
>>>longer exist in 3.0:
>>> apply, callable, coerce, dict.has_key, execfile, reduce,
>>>reload
>>>
>>
>>I see... what that document doesn't describe is the alternatives to be
>>used. And I see in that list a couple of functions that are probably used a
>>lot nowadays (callable, reduce, etc.).
>
>
> callable and reduce are rarely used, at least in code I've seen.
I do use callable(). Not everyday, for sure, but still often enough to
have to reimplement it when I'll switch to Py3K.
And while I rarely use it, I'll regret reduce().
> I
> would agree there will be a large number of programs that contain one
> or two calls to these functions, though. Certainly has_key will be
> the most common of those listed above (but trivial to fix). apply
> will be common in old code from the time of Python 1.5.2.
I still use it (in a somewhat deviant way) to define properties:
class MyClass(object):
def __init__(self, val):
self.val = val
@apply
def val():
def fget(self):
return self._val
def fset(self, val):
self._val = val
> execfile is
> perhaps more common that callable (?)
Never used it, never saw it used.
Bruno Desthuilliers
(snip)
> Instead of doing:
>
>
> if callable(function): function()
>
> you should do:
>
> try:
> function()
> except TypeError:
> pass
There are time where you may want to know if you have a callable without
calling it...
> That should work for most uses of callable(), but isn't quite the same.
> (What if function() has side-effects, or is expensive, and you want to
> determine if it is callable, but not actually call it _now_?)
Indeed.
The 'correct' replacement would be:
if hasattr(obj, '__call__'):
# it's a callable
but I don't find it so Pythonic to have to check for a __magic__ method.
>>Also, what is the replacement of reduce? I think I remember seeing
>>somewhere that lists comprehension would be (but also remember the
>>advise that reduce will be quicker).
>
>
> No, a list comprehension isn't equivalent to reduce(). There is no
> replacement for reduce().
>
> Guido has essentially done the functional equivalent of deciding that
> because he personally doesn't like for loops, they can and should be
> written out as:
>
> counter = 0
> while counter < len(sequence):
> item = sequence[counter]
> do_something_with_item
> counter += 1
>
> because it is "easier to read" than a for loop.
>
> And yes, if you think my example is extreme because nobody would be that
> stupid, that's the point: removing reduce() _is_ that stupid (and it is
> not often I say that about a language design decision by Guido!). Python
> can, and probably will, get away with it only because reducing an iterable
> to a single value is an uncommon operation, and the most common uses of it
> (summing a sequence, finding the minimum and maximum) already have
> workable replacements.
>
> It's especially stupid because map() and filter(), both of which do have
> simple, easy to understand, completely functional replacements, are going
> to stay. reduce() doesn't, and it is going. That's nuts.
InMyArms(TM) !
(Guido, if you read us...)
Alex Popescu
> Steven D'Aprano a écrit :
> (snip)
>
>> Instead of doing:
>
>> if callable(function): function()
>>
>> you should do:
>>
>> try:
>> function()
>> except TypeError:
>> pass
>
> There are time where you may want to know if you have a callable
> without calling it...
>
>> That should work for most uses of callable(), but isn't quite the
>> same. (What if function() has side-effects, or is expensive, and you
>> want to determine if it is callable, but not actually call it _now_?)
>
> Indeed.
>
> The 'correct' replacement would be:
>
> if hasattr(obj, '__call__'):
> # it's a callable
>
> but I don't find it so Pythonic to have to check for a __magic__
> method.
>
>
It looks like Python devs have decided it is Pythonic, because it is
already in the PEP. Same for execfile and the others. And I am not
arguing pro or con. I was just wondering about the reasons, as I haven't
been able to find anything (neither here nor on the dev group).
bests,
Bruno Desthuilliers
> Bruno Desthuilliers <bdesth.qu...@free.quelquepart.fr> wrote in
> news:46b0ed0b$0$2257$426a...@news.free.fr:
>> if hasattr(obj, '__call__'):
>> # it's a callable
>>
>> but I don't find it so Pythonic to have to check for a __magic__
>> method.
>
> It looks like Python devs have decided it is Pythonic, because it is
> already in the PEP.
I do know, and I disagree with this decision.
FWIW, repr(obj) is mostly syntactic sugar for obj.__repr__(),
getattr(obj, name) for obj.__getattr__(name), type(obj) for
obj.__class__ etc... IOW, we do have a lot of builtin functions that
mostly encapsulate calls to __magic__ methods, and I definitively don't
understand why this specific one (=> callable(obj)) should disappear. I
usually have lot of respect for Guido's talent as a language designer
(obviously since Python is still MFL), but I have to say I find this
particular decision just plain stupid. Sorry.
Alex Martelli
wrote:
> Alex Popescu a écrit :
> > Bruno Desthuilliers <bdesth.qu...@free.quelquepart.fr> wrote in
> > news:46b0ed0b$0$2257$426a...@news.free.fr:
> (snip)
> >> if hasattr(obj, '__call__'):
> >> # it's a callable
> >>
> >> but I don't find it so Pythonic to have to check for a __magic__
> >> method.
> >
> > It looks like Python devs have decided it is Pythonic, because it is
> > already in the PEP.
>
> I do know, and I disagree with this decision.
>
> FWIW, repr(obj) is mostly syntactic sugar for obj.__repr__(),
> getattr(obj, name) for obj.__getattr__(name), type(obj) for
> obj.__class__ etc... IOW, we do have a lot of builtin functions that
> mostly encapsulate calls to __magic__ methods, and I definitively don't
> understand why this specific one (=> callable(obj)) should disappear. I
Maybe because it DOESN'T "encapsulate a call" to a magic method, but
rather the mere check for the presence of one?
> usually have lot of respect for Guido's talent as a language designer
> (obviously since Python is still MFL), but I have to say I find this
> particular decision just plain stupid. Sorry.
The mere check of whether an object possesses some important special
method is best accomplished through the abstract-base-classes machinery
(new in Python 3.0: see <http://www.python.org/dev/peps/pep-3119/>). At
this time there is no Callable ABC, but you're welcome to argue for it
on the python-3000 mailing list (please do check the archives and/or
check privately with the PEP owner first to avoid duplication).
Alex
Bruno Desthuilliers
> Bruno Desthuilliers <bruno.42.de...@wtf.websiteburo.oops.com>
> wrote:
>
>> Alex Popescu a écrit :
>>> Bruno Desthuilliers <bdesth.qu...@free.quelquepart.fr> wrote in
>>> news:46b0ed0b$0$2257$426a...@news.free.fr:
>> (snip)
>>>> if hasattr(obj, '__call__'):
>>>> # it's a callable
>>>>
>>>> but I don't find it so Pythonic to have to check for a __magic__
>>>> method.
>>> It looks like Python devs have decided it is Pythonic, because it is
>>> already in the PEP.
>> I do know, and I disagree with this decision.
>>
>> FWIW, repr(obj) is mostly syntactic sugar for obj.__repr__(),
>> getattr(obj, name) for obj.__getattr__(name), type(obj) for
>> obj.__class__ etc... IOW, we do have a lot of builtin functions that
>> mostly encapsulate calls to __magic__ methods, and I definitively don't
>> understand why this specific one (=> callable(obj)) should disappear. I
>
> Maybe because it DOESN'T "encapsulate a call" to a magic method, but
> rather the mere check for the presence of one?
Yes, true. But it doesn't make such a big difference IMHO : it's about
decoupling API (the builtin funcs) from implementation (accessing |
calling | checking for the presence of a given implementation attribute).
>> usually have lot of respect for Guido's talent as a language designer
>> (obviously since Python is still MFL), but I have to say I find this
>> particular decision just plain stupid. Sorry.
>
> The mere check of whether an object possesses some important special
> method is best accomplished through the abstract-base-classes machinery
> (new in Python 3.0: see <http://www.python.org/dev/peps/pep-3119/>). At
> this time there is no Callable ABC, but you're welcome to argue for it
> on the python-3000 mailing list (please do check the archives and/or
> check privately with the PEP owner first to avoid duplication).
I'll have a look. Thanks for the pointers.
Alex Popescu
al...@mac.com:
> Bruno Desthuilliers <bruno.42.de...@wtf.websiteburo.oops.com>
> wrote:
>
>> Alex Popescu a écrit :
>
> [... snip ...]
>
>
> The mere check of whether an object possesses some important special
> method is best accomplished through the abstract-base-classes machinery
> (new in Python 3.0: see <http://www.python.org/dev/peps/pep-3119/>). At
> this time there is no Callable ABC, but you're welcome to argue for it
> on the python-3000 mailing list (please do check the archives and/or
> check privately with the PEP owner first to avoid duplication).
>
This PEP sounds very similar to Java interfaces (well, as an enhanced
version as you are allowed to programmatically control this inheritance
chain). Interesting approach. And now I can agree that the removal of
callable() makes sense!
Thanks a lot for the pointer Alex.
./alex
--
.w( the_mindstorm )p.
>
> Alex
>