I suggest that Python should raise warnings.RuntimeWarning (or similar?)
when a function is defined with a default argument consisting of a list,
dict or set. (This is not meant as an exhaustive list of all possible
mutable types, but as the most common ones that I expect will trip up
newbies.) The warning should refer to the relevant FAQ or section in the
docs.
What do people think?
--
Steven
I just suggested a documentation enhancement in one of the plethora of
threads... so I'm certainly +1 for any enhancement in this area.
Diez
I don't see a chance for your proposal. How are you going to detect
mutable objects? Custom types can be mutable as well as immutable.
Christian
> Steven D'Aprano wrote:
>> I suggest that Python should raise warnings.RuntimeWarning (or similar?)
>> when a function is defined with a default argument consisting of a list,
>> dict or set. (This is not meant as an exhaustive list of all possible
>> mutable types, but as the most common ones that I expect will trip up
>> newbies.) The warning should refer to the relevant FAQ or section in the
>> docs.
>>
>> What do people think?
-0 from me. I'd rather feature it more prominently in the tutorial, a
section "The five most common pitfalls" or something like that.
> I don't see a chance for your proposal. How are you going to detect
> mutable objects? Custom types can be mutable as well as immutable.
A check at compilation time for list literals would catch 90 % of the cases.
The warning would be targeted at newbies after all. It might still be a
source of confusion when they try to import someone else's code that uses
mutable defaults intentionally.
Peter
So maybe a command line option can be added to Python3 ( -
newbie ? :-) ) that just switches on similar warnings, to help newbies
(in schools, where there's a teacher that encourages to always use
that command line option) avoid some of the most common traps.
Bye,
bearophile
Or maybe bundle pychecker with idle?
$ cat tmp.py
def test(x, a=[]):
a.append(x)
return a
for i in range(5):
print test(i)
$ pychecker tmp.py
Processing tmp...
[0]
[0, 1]
[0, 1, 2]
[0, 1, 2, 3]
[0, 1, 2, 3, 4]
Warnings...
tmp.py:2: Modifying parameter (a) with a default value may have unexpected
consequences
Though it might be interesting to ask a newbie what he expects when warned
of "unexpected consequences" ;)
Peter
I like the idea of calling it to your attention. +1.
If a warning, you should be able to silence it with an annotation,
decorator, or a module-level flag.
Or perhaps, require it be declared explicit, and make it an error.
def test(x, a=defmut([])):
Python raises an actual error unless default arguments are known
immutable or instances of 'defmut'.
-1
People that have worked through the tutorial, something everyone should
do when they're starting out, will find this explicitly discussed. See
http://docs.python.org/tut/node6.html#SECTION006710000000000000000
People that just skim the surface get stung -- sorry.
Emile
Why should we add a new method?
Seriously, why should Python be cluttered and slowed down to warn about
mutable arguments. It's neither a design flaw nor a surprising feature
*ONCE* you have understood how functions work. I agree that the behavior
may be surprising for a newbie but it's logical once you got the big
picture.
Christian
But obviously enough, it's not emphazized enough. Even if the
interpreter isn't touched, at least the docs should be.
Diez
Well, that might be why it's the docs the way it is, and the only place
in the docs that carries the "Important warning" label, and in bold at that.
Further, the tutorial is the first link on the python for programmers
page, and on the non-programmers page it's the last link, so presumably
any non-programmer continuing on is pointed to the next step.
... so as to emphasized enough, how more?
Emile
Hmmm, OK -- mutable defaults, integer division, name mangling...
I'd think decimal precision is more a general problem than a python
problem, but still one that throws newbies...
Any other ideas for gotcha's (as opposed to faqs)?
Emile
It is not meant to catch all conceivable mutable types, just the three
most likely to surprise newbies.
The analogy is with the sum() built-in, which raises an exception if you
use it to add strings. (Personally, I think it should raise a warning,
not an exception, and allow people to write slow code if they want.)
Python doesn't attempt to detect every single type that leads to O(n**2)
behaviour in sum(), just the one most likely to trip up newbies.
But if people decide that it is important to warn on any conceivable
mutable type, the easy way is to approach the question from the other
direction. If the default value *isn't* a string, int, float, None or
frozenset, assume it's mutable.
Personally I don't think it's important to catch every case. After all,
it's just a warning, not an error.
--
Steven
> Steven D'Aprano wrote:
>> I suggest that Python should raise warnings.RuntimeWarning (or
>> similar?) when a function is defined with a default argument consisting
>> of a list, dict or set. (This is not meant as an exhaustive list of all
>> possible mutable types, but as the most common ones that I expect will
>> trip up newbies.) The warning should refer to the relevant FAQ or
>> section in the docs.
>>
>> What do people think?
>>
>>
>>
> -1
>
> People that have worked through the tutorial, something everyone should
> do when they're starting out,
I never worked through the Python tutorial. I bought a book. By the time
I even knew the tutorial existed, I had finished the book and spent six
months programming in Python and the tutorial was far too basic for me.
I'm sure there are thousands of other Python developers who have had
similar experiences. You can't expect every beginner's book on Python
programming to feature a discussion on mutable defaults.
Perhaps Python should print a warning when you start up: "If you haven't
worked through the tutorial, you are not allowed to ask questions about
surprising behaviour".
> will find this explicitly discussed. See
>
> http://docs.python.org/tut/node6.html#SECTION006710000000000000000
>
> People that just skim the surface get stung -- sorry.
You assume that people will read the tutorial and then immediately trip
over a real life example. There's a lot of stuff in the tutorial and it
doesn't all stick the first time you read it. Just recently, perhaps a
week or so ago, we had a case of a fellow who had been programming in
Python for many years before he stumbled across this behaviour. (I forget
the name of the thread, but I'm sure you can find it.) It's not just
noobs who trip over this.
--
Steven
Augmented assignment: x ?= y is not always the same as x = x ? y.
Repeated string addition can be very slow. For that matter, so can list
addition.
Inserting at the beginning of lists is slow.
Everything about unicode is a Gotcha! *wink*
Raw strings are not designed for Windows paths, they're designed for
regexes. Consequently, you can't write the following:
r'C:\dir\'
list.sort() and list.reverse() return None.
sorted() returns a list, but reversed() returns an iterator.
urllib2.urlopen() will automatically detect the proxy in your environment
and use that. That's usually a feature, but sometimes it can be a gotcha.
urllib2 doesn't work well with some HTTPS proxies. This is, I believe, a
known bug, but until it is fixed, it can be a gotcha.
--
Steven
My "favorite": comparisons between disparate types are allowed by
default. Thankfully fixed in 3.0.
George
-1
There's nothing questionable about using a mutable default argument,
as long as you don't mutate it. Python shouldn't raise a warning just
because something *might* be due to a misunderstanding.
I usefully use mutable default arguments all the time. Most commonly
in situations like this:
def f(data,substitutions = {}):
...
name = data['name']
obj.name = substitutions.get(name,name)
...
-0 on adding a warning that's disbaled by default.
Carl Banks
+1 on this.
It seems an obvious think to add to a lint-like tool rather than
burdening core Python.
- Paddy.
> On Aug 22, 10:42 am, Steven D'Aprano <st...@REMOVE-THIS-
> cybersource.com.au> wrote:
>> Sometimes it seems that barely a day goes by without some newbie, or
>> not- so-newbie, getting confused by the behaviour of functions with
>> mutable default arguments. No sooner does one thread finally, and
>> painfully, fade away than another one starts up.
>>
>> I suggest that Python should raise warnings.RuntimeWarning (or
>> similar?) when a function is defined with a default argument consisting
>> of a list, dict or set. (This is not meant as an exhaustive list of all
>> possible mutable types, but as the most common ones that I expect will
>> trip up newbies.) The warning should refer to the relevant FAQ or
>> section in the docs.
>>
>> What do people think?
>
> -1
>
> There's nothing questionable about using a mutable default argument, as
> long as you don't mutate it.
There's nothing questionable about using a mutable default argument, so
long as you know what behaviour to expect. I too use that behaviour, I
like that behaviour, and I'm tired of people who want it "fixed".
Nevertheless, it is surprising to many people. My aim is to make it a
little less surprising.
> Python shouldn't raise a warning just
> because something *might* be due to a misunderstanding.
That's one opinion.
As I've eluded to in an early post, I don't believe Python should refuse
to perform an operation just because it might be slow. Nevertheless,
that's precisely what the sum() function does. I'm suggesting a warning
rather than an exception, but other than that, I suggest that there's
precedence to what I am suggesting.
--
Steven
I think that is an excellent idea! In fact why can't pychecker be
included in the standard distribution? I'd love it if compilation was
done with pychecker checking by default. Python could definitely use a
-Wall mode.
--
mvh Björn
1. When you print spurious warnings, the overall effectiveness of the
warning system is diminished. People start to ignore them, either by
disabling them or by mentally tuning out. This in turn makes people
less likely to notice if a real warning is printed.
When you print a warning, you better be %99.9 sure that it's something
worth warning about, otherwise you are doing more harm than good.
(Story time: I once worked on a system that displayed warnings to jet
fighter pilots. Our requirements were not to show the pilot a warning
unless the airplane actually tries something and fails, even if the
computer is absolutely sure that it would fail. Ex: if computer knows
for sure the engine can't produce more than (say) 50% rated thrust,
the pilot does not get a warning unless he actually requests more than
50% thrust. The reason is for this, according to the senior engineers
on the team, was that pilots would start to ignore the warning lights
REALLY FAST.)
2. It's rude to be presumptuous, which is what the compiler would be
if it printed this warning.
Carl Banks
-1. I think, as many people before me has said, we should treat
programmers as an adult[1], and doesn't try to babysit them by giving
a warning, or worse as an exception. But I do agree that python should
have something like -Wall, where programmers are warned of all
potential problems.
[1] Anyway, you spent your time being a newbie only for a very short
time.
Question: what is real warning?
Don't MAKE ME have to tell you AGAIN !!!!
--
Stanley C. Kitching
Human Being
Phoenix, Arizona
Two black eyes. Haa haa. My question comes from: "less likely to
notice if a real warning is printed." And this one is a real one, is
my point.