popkey() method for dictionaries?
Fernando Pérez
I've often found myself needing what I've called a popkey() function. Similar
to the existing popitem() method, but which allows me to specify _which_ item
I need via a key. This is what I've implemented (as a function), the code is
trivial:
#----------------------------------------------------------------------------
class NotGiven: pass
def popkey(dct,key,default=NotGiven):
"""Return dct[key] and delete dct[key]. dct is modified in-place.
If default is given, return it if dct[key] doesn't exist, otherwise raise
KeyError. """
try:
val = dct[key]
except KeyError:
if default is NotGiven:
raise
else:
return default
else:
del dct[key]
return val
#----------------------------------------------------------------------------
Do people think that this would be a worthwhile thing to have in the standard
python dicts (as a method)? Basically I'm asking for opinions on whether it's
a good/bad idea, and if it's considered good it could be sent to the
developers.
It could even be implemented as an extension to the existing popitem() method.
Currently popitem() doesn't take any arguments, so the above 'key' argument
could be made optional.
What do people think? Good/bad? Useless to most?
Cheers,
f.
Erik Max Francis
> It could even be implemented as an extension to the existing popitem()
> method.
> Currently popitem() doesn't take any arguments, so the above 'key'
> argument
> could be made optional.
>
> What do people think? Good/bad? Useless to most?
Seems like a reasonable extension, but I can't imagine it would come
into all that much use.
--
Erik Max Francis / m...@alcyone.com / http://www.alcyone.com/max/
__ San Jose, CA, USA / 37 20 N 121 53 W / &tSftDotIotE
/ \ I only drink to make other people seem interesting.
\__/ George Jean Nathan
CSBuddy / http://www.alcyone.com/pyos/csbuddy/
A Counter-Strike server log file monitor in Python.
Janto Dreijer
Is this what you're looking for?
I love that time machine :-)
Fernando Pérez
Nice :) It's almost what I suggested, but lacks the -optional- argument for a
default value. This really seems like an oversight to me, and it violates the
principle of least surprise: d.get() has an optional default, but d.pop()
does not? They are functionally almost identical, except that pop() does a
delete afterwards. Why do these two methods have non-uniform interfaces??
I'd like to hear from one of the powers-that-be on this one. Or should a SF
bug be filed against it?
Cheers,
f.
Neil Schemenauer
> I'd like to hear from one of the powers-that-be on this one. Or should a SF
> bug be filed against it?
It seems like an oversight. Here's the original patch:
http://sf.net/tracker/?group_id=5470&atid=305470&func=detail&aid=539949
Feel free to file a bug. Cheers,
Neil
Fernando Pérez
> It seems like an oversight. Here's the original patch:
>
> http://sf.net/tracker/?group_id=5470&atid=305470&func=detail&aid=539949
>
> Feel free to file a bug. Cheers,
Just filed:
[ 639806 ] Optional argument for dict.pop() method
at:
https://sourceforge.net/tracker/index.php?func=detail&aid=639806&group_id=5470&atid=105470
Sorry but I just saw that my proof-of-concept code in python got squashed by
SF. I didn't know that SF removed all leading whitespace, hence destroying
indentation. Here it is again for reference:
#----------------------------------------------------------------------------
class NotGiven: pass
def popkey(dct,key,default=NotGiven):
"""Return dct[key] and delete dct[key]. dct is modified in-place.
If default is given, return it if dct[key] doesn't exist, otherwise raise
KeyError. """
try:
val = dct[key]
except KeyError:
if default is NotGiven:
raise
else:
return default
else:
del dct[key]
return val
#----------------------------------------------------------------------------
Cheers,
f.
Raymond Hettinger
>
> at:
>
https://sourceforge.net/tracker/index.php?func=detail&aid=639806&group_id=54
70&atid=105470
>
> Sorry but I just saw that my proof-of-concept code in python got squashed
by
> SF. I didn't know that SF removed all leading whitespace, hence destroying
> indentation. Here it is again for reference:
Do you have use cases demonstrating the value of a default rather than an
exception?
Also, discuss why similarity with dict.get() applies instead of symmetry
with list.pop() or dict.popitem().
Raymond Hettinger
Fernando Pérez
> Do you have use cases demonstrating the value of a default rather than an
> exception?
Sure. First, on principle: the exception option is still there, if no default
is provided. What giving a default buys you is not having to trap the
exception yourself. If you want a missing key to generate an exception,
simply don't give a default and that's it. Since it doesn't change the
existing semantics (the current form doesn't take any arguments, so there
can't be any confusion), I think it's a good addition.
But you asked for code. Here's an example from a wrapper which needs to filter
out a keyword dictionary given to a function before passing it downstream. It
needs to remove keywords which won't be understood by the downstream
function, but it knows how to make a default decision if the keyword wasn't
given:
# Filter out other options the original plot doesn't know
hardcopy = popkey(keyw,'hardcopy',psargs['filename'] is not None)
titles = popkey(keyw,'titles',0)
This uses my popkey() function, with the new method it would be simply
# Filter out other options the original plot doesn't know
hardcopy = keyw.pop('hardcopy',psargs['filename'] is not None)
titles = keyw.pop('titles',0)
if my suggestion were accepted. I can always live with my popkey() function
instead, if the crippled version is left in the language :) What I _won't_ do
is use the crippled pop() and wrap things everywhere with explicit try/except
blocks. In the end that only hurts readaility and creates bloat.
This is part of the Gnuplot wrappers in IPython, if you want to see the full
code (or I can give more context). IPython lives at
http://www-hep.colorado.edu/~fperez/ipython
> Also, discuss why similarity with dict.get() applies instead of symmetry
> with list.pop() or dict.popitem().
- list.pop: I wasn't looking when that was written :) But I could argue
similarly that an optional default argument would be a sensible, useful
addition. No semantic break for existing code, and it saves people setting up
try/except traps.
- dict.popitem: this one I'm ok with not having a default value, since it is
meant to traverse the dict in essentially random order. Adding a default
value would be like having a dict_with_default kind of object, which is
definitely different from a regular python dict. popitem() returns a k/v pair
and is typically used for exhausting a dict destructively, so it makes a lot
of sense to break at the end.
But pop(key) is much more specific, so I think it is sensible to be able to
control its behavior more closely. Just like dict.get(), which allows
straight through the function interface one to control what happens to
missing keys. Semantically pop(key) is almost like get(key), with the
difference that it deletes the k/v entry from the dict. It seems only natural
then that pop() would support the same default arg behavior that get() has.
Cheers,
f.
Alex Martelli
>> [ 639806 ] Optional argument for dict.pop() method
>>
>> at:
>>
>
https://sourceforge.net/tracker/index.php?func=detail&aid=639806&group_id=54
> 70&atid=105470
>>
>> Sorry but I just saw that my proof-of-concept code in python got squashed
> by
>> SF. I didn't know that SF removed all leading whitespace, hence
>> destroying indentation. Here it is again for reference:
>
> Do you have use cases demonstrating the value of a default rather than an
> exception?
I'm not the OP, but: while I still have no use of dict.pop in my code (as
I need to keep my code compating with Python versions that lack dict.pop),
those areas of my code where I'd LIKE to eventually insert dict.pop are
as follows:
-- some base class X has methods that accept a lot of keyword arguments --
e.g. X can typically be some GUI widget
-- I subclass X with my own Y to specialize some behavior, and typically
want to add some specialized options (keyword arguments) of mine
So my code typically runs something like:
class Y(X):
specialized_options = {'foo': 23, 'bar': 42, 'baz': 66}
def __init__(self, **kwds):
# get specialized opts out of kwds:
specialized_options = {}
for opt in self.specialized_options:
try:
val = kwds[opt]
except KeyError:
val = self.specialized_options[opt]
else:
del kwds[opt]
specialized_options[opt] = val
X.__init__(self, **kwds)
for opt, val in specialized_options.items():
setattr(self, opt, val)
or obvious variants such as starting with
specialized_options = self.specialized_options.copy()
and/or using "if opt in kwds" rather than the try/except (not
much in it one way or another), and/or looping with "for opt, val"
on self.specialized_options.items() [or .iteritems()], etc. (Of
course I could use a list of pairs just as well as a dict for
specialized_options, etc, etc -- just pointing all of these obvious
and irrelevant issues in the faint hope that followups won't detour
on them but rather stay focused on the .pop issue...:-).
Now, if I could use dict.pop _with exception behavior only_ that might
become (repeating the body of __init__ only):
specialized_options = {}
for opt in self.specialized_options:
try:
val = kwds.pop(opt)
except KeyError:
val = self.specialized_options[opt]
specialized_options[opt] = val
X.__init__(self, **kwds)
for opt, val in specialized_options.items():
setattr(self, opt, val)
this IS a slight enhancement -- I save the else clause in the
try/except in the for loop's body. But, rather slight: I still
do need the try/except, or equivalently an "if opt in kwds" test.
If I could give dict.pop a default value, then I would need
no try/except nor any membership-test:
specialized_options = {}
for opt in self.specialized_options:
val = kwds.pop(opt, self.specialized_options[opt])
specialized_options[opt] = val
X.__init__(self, **kwds)
for opt, val in specialized_options.items():
setattr(self, opt, val)
I think this is a substantial simplification. I might or might
not bother doing the refactoring to use kwds.pop with the exception,
as the enhancement is so slight, but this one I'd jump for. Further,
removing the need to handle the exceptions would enable me to use
a list comprehension instead if I so wished:
specialized_options = [ (opt, kwds.pop(opt, val))
for opt, val in self.specialized_options.items() ]
replacing the first 4 statements in the latest snippet -- I'd definitely
not make specialized_options a dict in this case, as a list of pairs is
obviously simpler to build -- so the final for would be:
for opt, val in specialized_options:
setattr(self, opt, val)
instead.
Not sure if I'd go for this one. Hmmm, perhaps. But I'd sure
like to have the option, anyway.
> Also, discuss why similarity with dict.get() applies instead of symmetry
> with list.pop() or dict.popitem().
popitem has no symmetry whatsoever since it takes no argument, of course.
So, I must be missing something: WHAT symmetry?
lists have no .get method which lets you supply a default. Maybe they
should, because I sure DO code very often:
if 0 <= i < len(somelist):
xx = somelist[i]
else:
xx = somedefault
or variants thereof where i's allowed to be negative down to -len(somelist),
and being able to code "xx = somelist.get(i, somedefault)", just as I could
with a dictionary instead of a list, might be quite handy.
list.pop can take 0 or 1 argument. If you're claiming there is symmetry,
you must be proposing to have dict.pop take 0 or 1 argument too -- but
that's not the case (it might have been a possible design, but instead a
separate .popitem method was introduced for dictionaries). Since there is
no symmetry anyway, there is no argument for symmetry between list.pop and
dict.pop.
I'm not sure what's being proposed for somedict.pop(x) (called with just
one argument) to do when "x not in somedict" -- I'd like that to raise an
exception, NOT meekly return None. So, I'm not pumping for strong symmetry
between dict.get and dict.pop -- not sure if the proposal as tabled is.
The point is that, when you're simply accessing an item, you can already
choose between:
somedict[x]
when you want x's absence to raise an exception, versus
somedict.get(x)
when you want x's absence to meekly return None instead. Both behaviors
are useful in different circumstances (i.e. whether you expect or not that
absence, or whether in the absence case you must perform some very
different processing). I would NOT like it if there was no handy way to
say "delete-and-return somedict[x] which SHOULD be there so please give me
an exception if it isn't" which is probably more common than the cases
where you fully expect it to possibly not be there.
The analogy rather than symmetry that I see is with builtin getattr: if I
don't supply a default I get an exception if the attribute is absent, but I
also get the opportunity to supply a default and avoid the exception.
Alex
John Hunter
Fernando> Sure. First, on principle: the exception option is still
Fernando> there, if no default is provided. What giving a default
Fernando> buys you is not having to trap the exception
Fernando> yourself. If you want a missing key to generate an
Fernando> exception, simply don't give a default and that's
Fernando> it.
But then you break the symmetry with get, which provides None as a
default if no default is provided. Aren't you now back in danger of
violating the principle of least surprise?
John Hunter
Fernando Pérez
Mmmh. I hadn't used get without default in a long time, and I'd forgotten its
not-found return was a None instead of an exception. I would argue _that_
violates the 'explicit is better than implicit' rule of python, but it's a
bit late to change that one :)
In general, since these are all wrappers around dict[key] access operations,
and at the most basic level such an access to a non-existent key _will_
trigger an exception, I'd argue that the default behavior of all these
functions (get, pop, popitem) should be to raise an exception. In this way
they would most naturally mimic the underlying semantics, without magic
behaviors. On the other hand, I think it is _useful_ syntactic convenience to
have a way of explicitly overriding the exception triggering, for the reasons
Alex Martelli and I argued elsewhere in this tread.
Unfortunately these functions (get/pop/popitem) have already been written with
(it seeems to me) inconsistent interfaces, and we'll just get used to keeping
track of who does what with post-it notes.
Cheers,
f.
Alex Martelli
...
> key _will_ trigger an exception, I'd argue that the default behavior of
> all these functions (get, pop, popitem) should be to raise an exception.
Practicality beats purity, and it would be no use for somedict.get(key) to
be just alternative syntax for somedict[key]; while having it return None
when key not in somedict is useful to enable, for example, such use
as map(somedict.get, potentialkeys).
> Unfortunately these functions (get/pop/popitem) have already been written
> with (it seeems to me) inconsistent interfaces, and we'll just get used to
Yes, somewhat, but I think they're each pragmatic enough that keeping track
is no major chore (IMHO).
Alex