[Python-ideas] keywording prohibited

7 views
Skip to first unread message

spir

unread,
Apr 23, 2009, 3:48:35 AM4/23/09
to *python-ideas
Hello,

I have noticed a rather strong and nearly systematic opposition to (new) keywords. Cannot really figure out why. Would someone clearly expose the reasoning behind keeping the keyword set as small as possible? (I thought at not preventing users to use the same words as names, but this reason does not seem to hold. On the contrary: non-keyword builtins bite painfully!)
Could not find references on the topic -- probably because a search containing "keyword" returns loads of unrelated stuff.

Denis
------
la vita e estrany
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas

Ben Finney

unread,
Apr 23, 2009, 4:11:48 AM4/23/09
to python...@python.org
spir <denis...@free.fr> writes:

> I have noticed a rather strong and nearly systematic opposition to
> (new) keywords. Cannot really figure out why. Would someone clearly
> expose the reasoning behind keeping the keyword set as small as
> possible?

Because a programming language whose syntax and keyword set can be
easily contained all in one's head, concurrently with all the other
things that one must keep thinking about while programming, is superior
to one for which that's more difficult. Increasing the size of the
language (its keywords or other syntax) moves it in that dimension from
easy to difficult.

Sometimes that's worth the cost; but the cost is significant, so the
resistance must be strong.

--
\ “Broken promises don't upset me. I just think, why did they |
`\ believe me?” —Jack Handey |
_o__) |
Ben Finney

Nick Coghlan

unread,
Apr 23, 2009, 7:50:52 AM4/23/09
to spir, *python-ideas
spir wrote:
> Hello,
>
> I have noticed a rather strong and nearly systematic opposition to
> (new) keywords. Cannot really figure out why. Would someone clearly
> expose the reasoning behind keeping the keyword set as small as
> possible? (I thought at not preventing users to use the same words as
> names, but this reason does not seem to hold. On the contrary:
> non-keyword builtins bite painfully!)

However, using builtin names as attribute or method names is often quite
useful and sensible. You can't do that with keywords - those are
disallowed everywhere other than in the syntactic constructs that rely
on them. You can see the ugly workarounds that are needed in the case of
a couple of existing keywords like 'class' (where people have to use
'cls', 'class_' or 'klass' instead) and 'assert' (generally replaced
with either 'assert_' or 'assertTrue').

In addition to the above and to what Ben said, every new keyword also
means going through the whole __future__ and DeprecationWarning dance in
order to warn people about the introduction of the new keyword without
breaking existing code. Any users whose code triggers those warnings
will have to expend additional effort to port their application to the
Python version where the new keyword is enabled by default.

That last point is enough of a reason to dislike the idea of new
keywords purely from the point of view of aggregate development effort
(i.e. not just effort from the core devs, but the porting effort from
all the affected third part developers as well).

Cheers,
Nick.

--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
---------------------------------------------------------------

Aahz

unread,
Apr 23, 2009, 8:53:54 AM4/23/09
to python...@python.org
On Thu, Apr 23, 2009, spir wrote:
>
> I have noticed a rather strong and nearly systematic opposition to
> (new) keywords. Cannot really figure out why. Would someone clearly
> expose the reasoning behind keeping the keyword set as small as
> possible? (I thought at not preventing users to use the same words
> as names, but this reason does not seem to hold. On the contrary:
> non-keyword builtins bite painfully!)

How do non-keyword builtins bite?

Also, consider that every single non-Python syntax checking mechanism
must be updated (e.g. vim/emacs syntax highlighter).
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/

"If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur." --Red Adair

Greg Ewing

unread,
Apr 23, 2009, 6:19:00 PM4/23/09
to spir, *python-ideas
spir wrote:

> I have noticed a rather strong and nearly systematic opposition to (new)
> keywords. Cannot really figure out why.

Every time a new keyword is added, it has the potential
to break the code of someone who is using it as a
variable name.

Since we place a high value on not breaking existing
code, we are naturally reluctant to add new keywords.

We do sometimes, but there has to be an extremely good
reason for it.

--
Greg

Ben Finney

unread,
Apr 23, 2009, 8:22:54 PM4/23/09
to python...@python.org
Greg Ewing <greg....@canterbury.ac.nz> writes:

> Since we place a high value on not breaking existing code, we are
> naturally reluctant to add new keywords.

This also works in the opposite direction.

Despite all the natural barriers to adding a new keyword, it is still
far easier to add a new keyword than to *remove* one from the language
if it later turns out to be problematic: some portion of working code
will be thereby broken, which is much more likely than the breakage
caused by adding it in the first place.

Hance, since it's far more difficult to go back (from complex to
simple), there must be great burden of proof for the benefit in moving
from simple to complex.

--
\ “Two hands working can do more than a thousand clasped in |
`\ prayer.” —Anonymous |
_o__) |
Ben Finney

Jan Claeys

unread,
May 2, 2009, 5:35:29 PM5/2/09
to python...@python.org
Op donderdag 23-04-2009 om 05:53 uur [tijdzone -0700], schreef Aahz:

> How do non-keyword builtins bite?

You can accidentally "re-define" or mask them (if you don't use a tool
that warns for that).


--
Jan Claeys

Aahz

unread,
May 2, 2009, 6:57:35 PM5/2/09
to python...@python.org
On Sat, May 02, 2009, Jan Claeys wrote:
> Op donderdag 23-04-2009 om 05:53 uur [tijdzone -0700], schreef Aahz:
>>
>> How do non-keyword builtins bite?
>
> You can accidentally "re-define" or mask them (if you don't use a tool
> that warns for that).

Given how long it has been since the original post, I think it's unfair
of you to cut this much context. What Denis Spir originally wrote was,
"On the contrary: non-keyword builtins bite painfully!" and my rejoinder
was written because of the extreme language.

Your issue is a valid one, but I have seen no cases where I would call
it "painful" contrasted with the pain of being unable to use a keyword
anywhere as a name or attribute.

"Typing is cheap. Thinking is expensive." --Roy Smith

spir

unread,
May 2, 2009, 9:16:33 PM5/2/09
to python...@python.org
Le Sat, 2 May 2009 15:57:35 -0700,
Aahz <aa...@pythoncraft.com> s'exprima ainsi:

> >> How do non-keyword builtins bite?
> >
> > You can accidentally "re-define" or mask them (if you don't use a tool
> > that warns for that).
>
> Given how long it has been since the original post, I think it's unfair
> of you to cut this much context.

Sorry for this.

> What Denis Spir originally wrote was,
> "On the contrary: non-keyword builtins bite painfully!" and my rejoinder
> was written because of the extreme language.

Still, Jan's reply is the right issue I had in mind.

> Your issue is a valid one, but I have seen no cases where I would call
> it "painful" contrasted with the pain of being unable to use a keyword
> anywhere as a name or attribute.

Depends on your pov. I consider the _possibility_ of using 'list', 'range' or 'type' as a name for totally custom thing, without even a warning, an issue, not a wishable feature. It _invents_ very hard to diagnose bugs. Painful they are. I cannot even figure out a debugging process that would point to the real source of the problem -- except for a sudden "eureka".
Actually, the issue is that precisely the non-protected build-in words are those so obvious variable names for data or funcs.
I have (painfully) learnt a reflex to use 'ranj' for range, 'Seq' for list, 'typ' for type, etc...
But maybe its me and I'm the only one ;-)

[There is an (unsatisfying) solution with some editors: just add all those word to the keyword set so that they will jump at your eyes when you pretend use them... But they also look like keywords when safely used as attributes. I would prefere a python warning mode switch.]
Strangely enough, I cannot imagine a use case for 'for' or 'in' or 'while' as names. while these ones are protected (indeed, I don't mean they shouldn't).

Denis
------
la vita e estrany

Adam Atlas

unread,
May 2, 2009, 10:28:47 PM5/2/09
to python-ideas

On 2 May 2009, at 21:16, spir wrote:
> I have (painfully) learnt a reflex to use 'ranj' for range, 'Seq'
> for list, 'typ' for type, etc...
> But maybe its me and I'm the only one ;-)

Isn't the convention to suffix a name with an underscore when it would
clash with a builtin? (range_, list_, type_, etc.)

I'm not sure how much I like that stylistically, but I've seen it used
a lot. (I think in some cases there are better alternatives -- e.g.
instead of naming a variable "seq" or "list_", I'd have the name
specify what it's a list *of*.)

Stephen J. Turnbull

unread,
May 3, 2009, 1:45:08 AM5/3/09
to spir, python...@python.org
spir writes:

> Depends on your pov. I consider the _possibility_ of using 'list',
> 'range' or 'type' as a name for totally custom thing, without even
> a warning, an issue, not a wishable feature. It _invents_ very hard
> to diagnose bugs.

If you want warnings, use pylint. It catches all of those.

On the other hand, Python's even-handed treatment of builtins,
standard library identifiers, and user-defined identifiers makes for a
great simplification in the rules for the language, and allows for
introspective usage:

# Here's something you can do for fun on a lazy Sunday afternoon.
the_artist_formerly_known_as_list = list
def list(thing):
print "Somebody called list!"
return the_artist_formerly_known_as_list(thing)

For most of us, the gain in simplicity and power is well worth the pain.

> Strangely enough, I cannot imagine a use case for 'for' or 'in' or
> 'while' as names. while these ones are protected (indeed, I don't
> mean they shouldn't).

Dunno about the other two, but in C I used to use the equivalent of

in = open("mind", "r")
text = in.readlines()

all the time.

Steven D'Aprano

unread,
May 3, 2009, 2:59:35 AM5/3/09
to python...@python.org
On Sun, 3 May 2009 07:35:29 am Jan Claeys wrote:
> Op donderdag 23-04-2009 om 05:53 uur [tijdzone -0700], schreef Aahz:
> > How do non-keyword builtins bite?
>
> You can accidentally "re-define" or mask them (if you don't use a
> tool that warns for that).

Cross out "accidentally", and the ability to shadow built-ins is a
feature, not a bug.

Anything that you do accidentally is a problem. You can accidentally
shadow a module, or one of your own functions or names. And while
Python has a rich set of built-ins, it isn't so many that learning them
is a burden. I would expect that the average newbie would accidentally
shadow a built-in once or twice, then never do it again.

(Especially if they hang around comp.lang.python and get yelled at every
time they do it.)


--
Steven D'Aprano

Steven D'Aprano

unread,
May 3, 2009, 3:03:48 AM5/3/09
to python...@python.org
On Sun, 3 May 2009 12:28:47 pm Adam Atlas wrote:

> Isn't the convention to suffix a name with an underscore when it
> would clash with a builtin? (range_, list_, type_, etc.)

That's one convention. Another is to name things alist, atype, astr,.
etc. If you need two of them, the obvious extension is blist, btype...

When writing *small* generic methods or functions, I'm also fond of
using L for list, x and y for floats, n or i for ints, and similar.
This needs to be handled with care, for obvious reasons.


> I'm not sure how much I like that stylistically, but I've seen it
> used a lot. (I think in some cases there are better alternatives --
> e.g. instead of naming a variable "seq" or "list_", I'd have the name
> specify what it's a list *of*.)

Well, this is Python, and we use duck-typing, so everything would need
to be list_of_string_like_instances or similar :)

But seriously, I tend to use plurals for that. If I have an argument
that takes a collection of widgets, say, I call it "widgets", and write
code like this:

for widget in widgets:
whatever()

I don't think there's much to be gained by calling it "list_of_widgets"
unless it really needs to be a list, and not any other collection type.

--
Steven D'Aprano

spir

unread,
May 3, 2009, 5:36:17 AM5/3/09
to python...@python.org
Le Sun, 3 May 2009 17:03:48 +1000,
Steven D'Aprano <st...@pearwood.info> s'exprima ainsi:

> e.g. instead of naming a variable "seq" or "list_", I'd have the name
> > specify what it's a list *of*.)
>
> Well, this is Python, and we use duck-typing, so everything would need
> to be list_of_string_like_instances or similar :)
>
> But seriously, I tend to use plurals for that. If I have an argument
> that takes a collection of widgets, say, I call it "widgets", and write
> code like this:
>
> for widget in widgets:
> whatever()

I use exactly the same convention.
And 100% agree with the fact that the Good Way is to use specific names that denote what use you have for the value, ie the concept the name carries /in your prog/.
But precisely sometimes the meaning is generic. Or so obvious from the context or application that it really looks stupid to over-specify (while I often do it anyway). Also, humans are lazy. [One may require a python programmer not to be lazy, but it will not buy much, and we still have to cope with the fact -- including for ourselves.]

In fact, I think issues about reusing builtin name are similar to the ones when bad-naming in general, including the fact that the developper is the first one trapped. But the consequences are potentially much worse and difficulty of debugging usually very great.

Denis
------
la vita e estrany

John Graham

unread,
May 4, 2009, 8:47:25 AM5/4/09
to Stephen J. Turnbull, python...@python.org
On Sun, May 3, 2009 at 12:45 AM, Stephen J. Turnbull <ste...@xemacs.org> wrote:
> spir writes:
>
>  > Depends on your pov. I consider the _possibility_ of using 'list',
>  > 'range' or 'type' as a name for totally custom thing, without even
>  > a warning, an issue, not a wishable feature. It _invents_ very hard
>  > to diagnose bugs.
>
> If you want warnings, use pylint.  It catches all of those.

Has anyone ever suggested a standard pylint, something that might be
distributed with the interpreter/libraries? As important as the
'look-and-feel' is to Python, it seems like an automated style-checker
/ 'you probably didn't meant to do that' engine would be a good idea
to introduce even newbies to the language instead of them having to
search it out.

-jg

Aahz

unread,
May 4, 2009, 9:10:24 AM5/4/09
to python...@python.org
On Mon, May 04, 2009, John Graham wrote:
>
> Has anyone ever suggested a standard pylint, something that might be
> distributed with the interpreter/libraries? As important as the
> 'look-and-feel' is to Python, it seems like an automated style-checker
> / 'you probably didn't meant to do that' engine would be a good idea
> to introduce even newbies to the language instead of them having to
> search it out.

It's been mentioned; I don't remember why it never got any traction,
whether because the authors didn't want to deal with it or nobody
bothered submitting a PEP (this definitely needs a PEP).

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

CTO

unread,
May 4, 2009, 4:05:21 PM5/4/09
to python...@python.org
> It's been mentioned; I don't remember why it never got any traction,
> whether because the authors didn't want to deal with it or nobody
> bothered submitting a PEP (this definitely needs a PEP).

I'll volunteer to put one together if somebody's willing to walk me
through the PEP process.

Geremy Condra

Aahz

unread,
May 4, 2009, 4:23:48 PM5/4/09
to python...@python.org
On Mon, May 04, 2009, CTO wrote:
>
>> It's been mentioned; I don't remember why it never got any traction,
>> whether because the authors didn't want to deal with it or nobody
>> bothered submitting a PEP (this definitely needs a PEP).
>
> I'll volunteer to put one together if somebody's willing to walk me
> through the PEP process.

Um, I think you trimmed too much of the quote -- what are you talking
about?

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

CTO

unread,
May 4, 2009, 5:07:40 PM5/4/09
to python...@python.org

On May 4, 4:23 pm, Aahz <a...@pythoncraft.com> wrote:
> On Mon, May 04, 2009, CTO wrote:
>
> >> It's been mentioned; I don't remember why it never got any traction,
> >> whether because the authors didn't want to deal with it or nobody
> >> bothered submitting a PEP (this definitely needs a PEP).
>
> > I'll volunteer to put one together if somebody's willing to walk me
> > through the PEP process.
>
> Um, I think you trimmed too much of the quote -- what are you talking
> about?

Apologies- and apparently the list has eaten my earlier reply. This
was
in reference to the discussion about distributing an automated style
checker with python.

Geremy Condra

Aahz

unread,
May 4, 2009, 7:15:33 PM5/4/09
to python...@python.org
On Mon, May 04, 2009, CTO wrote:
> Aahz:

>> On Mon, May 04, 2009, John Graham wrote:
>>>
>>> Has anyone ever suggested a standard pylint, something that might be
>>> distributed with the interpreter/libraries? As important as the
>>> 'look-and-feel' is to Python, it seems like an automated style-checker
>>> / 'you probably didn't meant to do that' engine would be a good idea
>>> to introduce even newbies to the language instead of them having to
>>> search it out.
>>
>> It's been mentioned; I don't remember why it never got any traction,
>> whether because the authors didn't want to deal with it or nobody
>> bothered submitting a PEP (this definitely needs a PEP).
>
> I'll volunteer to put one together if somebody's willing to walk me
> through the PEP process.

Sure! I suggest that you start by checking with the maintainers of
pychecker and pylint to find out how they feel about it.

Chris Rebert

unread,
May 4, 2009, 8:25:24 PM5/4/09
to Aahz, python...@python.org
On Mon, May 4, 2009 at 4:15 PM, Aahz <aa...@pythoncraft.com> wrote:
> On Mon, May 04, 2009, CTO wrote:
>> Aahz:
>>> On Mon, May 04, 2009, John Graham wrote:
>>>>
>>>> Has anyone ever suggested a standard pylint, something that might be
>>>> distributed with the interpreter/libraries?  As important as the

Wouldn't it be be better placed in the Tools/ directory rather than as
part of the stdlib?

Cheers,
Chris
--
http://blog.rebertia.com

John Graham

unread,
May 4, 2009, 8:30:03 PM5/4/09
to Chris Rebert, python...@python.org

Touche', salesman.
-Jg

Ben Finney

unread,
May 4, 2009, 8:57:08 PM5/4/09
to python...@python.org
Aahz <aa...@pythoncraft.com> writes:

> >> On Mon, May 04, 2009, John Graham wrote:
> >>> Has anyone ever suggested a standard pylint, something that might
> >>> be distributed with the interpreter/libraries? As important as the
> >>> 'look-and-feel' is to Python, it seems like an automated
> >>> style-checker / 'you probably didn't meant to do that' engine
> >>> would be a good idea to introduce even newbies to the language
> >>> instead of them having to search it out.
>

> Sure! I suggest that you start by checking with the maintainers of
> pychecker and pylint to find out how they feel about it.

The popularity of ‘pyflakes’ suggests it would be good to coordinate
with that project too. <URL:http://divmod.org/trac/wiki/DivmodPyflakes>
<URL:http://github.com/kevinw/pyflakes/>

--
\ “When in doubt tell the truth. It will confound your enemies |
`\ and astound your friends.” —Mark Twain, _Following the Equator_ |
_o__) |
Ben Finney

Aahz

unread,
May 4, 2009, 9:25:27 PM5/4/09
to python...@python.org
On Mon, May 04, 2009, Chris Rebert wrote:
>>>> On Mon, May 04, 2009, John Graham wrote:
>>>>>
>>>>> Has anyone ever suggested a standard pylint, something that might be
>>>>> distributed with the interpreter/libraries? ?As important as the

>
> Wouldn't it be be better placed in the Tools/ directory rather than as
> part of the stdlib?

Perhaps, but I don't think so -- you need a source tree to get Tools/
AFAIK, and if some form of lint gets blessed by incorporation into
Python, then it needs to be available for all standard installs.

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

CTO

unread,
May 5, 2009, 3:45:07 AM5/5/09
to python...@python.org

On May 4, 8:57 pm, Ben Finney <ben+pyt...@benfinney.id.au> wrote:


> Aahz <a...@pythoncraft.com> writes:
> > >> On Mon, May 04, 2009, John Graham wrote:
> > >>> Has anyone ever suggested a standard pylint, something that might
> > >>> be distributed with the interpreter/libraries? As important as the
> > >>> 'look-and-feel' is to Python, it seems like an automated
> > >>> style-checker / 'you probably didn't meant to do that' engine
> > >>> would be a good idea to introduce even newbies to the language
> > >>> instead of them having to search it out.
>
> > Sure!  I suggest that you start by checking with the maintainers of
> > pychecker and pylint to find out how they feel about it.
>
> The popularity of ‘pyflakes’ suggests it would be good to coordinate
> with that project too. <URL:http://divmod.org/trac/wiki/DivmodPyflakes>
> <URL:http://github.com/kevinw/pyflakes/>

I've sent emails to pylint, pychecker, pyflakes, and pep8.py. We'll
see
what comes out of that, but if anybody is aware of any alternative
emails
for any of the above, I should probably send something to those too-
the
one for pylint in particular doesn't look like it's been checked in a
while.

Geremy Condra

Terry Reedy

unread,
May 5, 2009, 4:55:24 AM5/5/09
to python...@python.org
Aahz wrote:
> On Mon, May 04, 2009, Chris Rebert wrote:
>>>>> On Mon, May 04, 2009, John Graham wrote:
>>>>>> Has anyone ever suggested a standard pylint, something that might be
>>>>>> distributed with the interpreter/libraries? ?As important as the
>> Wouldn't it be be better placed in the Tools/ directory rather than as
>> part of the stdlib?
>
> Perhaps, but I don't think so -- you need a source tree to get Tools/
> AFAIK, and if some form of lint gets blessed by incorporation into
> Python, then it needs to be available for all standard installs.

Regardless, the stdlib is for modules to import, tools for programs that
run.

Michael Foord

unread,
May 5, 2009, 5:29:45 AM5/5/09
to Aahz, python...@python.org


2009/5/5 Aahz <aa...@pythoncraft.com>

On Mon, May 04, 2009, CTO wrote:
> Aahz:
>> On Mon, May 04, 2009, John Graham wrote:
>>>
>>> Has anyone ever suggested a standard pylint, something that might be
>>> distributed with the interpreter/libraries?  As important as the
>>> 'look-and-feel' is to Python, it seems like an automated style-checker
>>> / 'you probably didn't meant to do that' engine would be a good idea
>>> to introduce even newbies to the language instead of them having to
>>> search it out.
>>
>> It's been mentioned; I don't remember why it never got any traction,
>> whether because the authors didn't want to deal with it or nobody
>> bothered submitting a PEP (this definitely needs a PEP).
>
> I'll volunteer to put one together if somebody's willing to walk me
> through the PEP process.

Sure!  I suggest that you start by checking with the maintainers of
pychecker and pylint to find out how they feel about it.


PyLint is huge and depends on a whole ream of support libraries. I think the logilab guys would probably rather maintain it themselves and I'm not sure it is really suitable for the standard library anyway.

PyFlakes and PyChecker are both very good; although neither does as much as PyLint. I'm not sure if either of the two does enough to enforce full PEP-8 compliance.

Michael Foord

 

--
Aahz (aa...@pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas



--
http://www.ironpythoninaction.com/


Jeremy Hylton

unread,
May 5, 2009, 8:42:44 AM5/5/09
to Michael Foord, python...@python.org
On Tue, May 5, 2009 at 5:29 AM, Michael Foord <fuzz...@gmail.com> wrote:
>
>
> 2009/5/5 Aahz <aa...@pythoncraft.com>
>>
>> On Mon, May 04, 2009, CTO wrote:
>> > Aahz:
>> >> On Mon, May 04, 2009, John Graham wrote:
>> >>>
>> >>> Has anyone ever suggested a standard pylint, something that might be
>> >>> distributed with the interpreter/libraries?  As important as the
>> >>> 'look-and-feel' is to Python, it seems like an automated style-checker
>> >>> / 'you probably didn't meant to do that' engine would be a good idea
>> >>> to introduce even newbies to the language instead of them having to
>> >>> search it out.
>> >>
>> >> It's been mentioned; I don't remember why it never got any traction,
>> >> whether because the authors didn't want to deal with it or nobody
>> >> bothered submitting a PEP (this definitely needs a PEP).
>> >
>> > I'll volunteer to put one together if somebody's willing to walk me
>> > through the PEP process.
>>
>> Sure!  I suggest that you start by checking with the maintainers of
>> pychecker and pylint to find out how they feel about it.
>
> PyLint is huge and depends on a whole ream of support libraries. I think the
> logilab guys would probably rather maintain it themselves and I'm not sure
> it is really suitable for the standard library anyway.

We use pylint at work for our own lint tool. It's been useful to us.
I'm not sure that I see much value in adding it or something like it
to the standard distribution. If there is a good lint tool that
supports PEP 8 and is available for download, I expect most python
developers would install it.

Jeremy

Nick Coghlan

unread,
May 5, 2009, 10:46:46 AM5/5/09
to Terry Reedy, python...@python.org
Terry Reedy wrote:
> Aahz wrote:
>> On Mon, May 04, 2009, Chris Rebert wrote:
>>>>>> On Mon, May 04, 2009, John Graham wrote:
>>>>>>> Has anyone ever suggested a standard pylint, something that might be
>>>>>>> distributed with the interpreter/libraries? ?As important as the
>>> Wouldn't it be be better placed in the Tools/ directory rather than as
>>> part of the stdlib?
>>
>> Perhaps, but I don't think so -- you need a source tree to get Tools/
>> AFAIK, and if some form of lint gets blessed by incorporation into
>> Python, then it needs to be available for all standard installs.
>
> Regardless, the stdlib is for modules to import, tools for programs that
> run.

We blur that line all the time though (cf. timeit, pdb, pydoc,
webbrowser, runpy, probably others).

So I'd agree with Aahz that if python-dev is going to bless something
along these lines, it should be something worthy of inclusion in the
standard lib itself rather than just being dropped into the Tools directory.

Cheers,
Nick.

--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
---------------------------------------------------------------

Terry Reedy

unread,
May 6, 2009, 12:23:28 AM5/6/09
to python...@python.org
Nick Coghlan wrote:
> Terry Reedy wrote:
>> Aahz wrote:
>>> On Mon, May 04, 2009, Chris Rebert wrote:
>>>>>>> On Mon, May 04, 2009, John Graham wrote:
>>>>>>>> Has anyone ever suggested a standard pylint, something that might be
>>>>>>>> distributed with the interpreter/libraries? ?As important as the
>>>> Wouldn't it be be better placed in the Tools/ directory rather than as
>>>> part of the stdlib?
>>> Perhaps, but I don't think so -- you need a source tree to get Tools/

My standard Windows 3.0.1 installation has Tools without the source
tree. It has 5 directories, including Scripts/, which has about 50 .py
files, including 2to3. If *that* can go there, I see no reason why a
syntax checker could not also.

>>> AFAIK, and if some form of lint gets blessed by incorporation into
>>> Python, then it needs to be available for all standard installs.
>> Regardless, the stdlib is for modules to import, tools for programs that
>> run.
>
> We blur that line all the time though (cf. timeit, pdb, pydoc,
> webbrowser, runpy, probably others).
>
> So I'd agree with Aahz that if python-dev is going to bless something
> along these lines, it should be something worthy of inclusion in the
> standard lib itself rather than just being dropped into the Tools directory.

timit, pdb, webbrowser, amd runpy are importable modules that *also*
have a command-line interface via "if __name__ == '__main__': ...".
Pydoc might be also, but I could not be sure from the doc. If the
checker were written similarly, so that it could be imported and used
from within a program to check another file, then I would agree that
stdlib would be a place for it.

tjr

CTO

unread,
May 6, 2009, 12:48:46 AM5/6/09
to python...@python.org
[snip]

> My standard Windows 3.0.1 installation has Tools without the source
> tree.  It has 5 directories, including Scripts/, which has about 50 .py
> files, including 2to3. If *that* can go there, I see no reason why a
> syntax checker could not also.
>
> >>> AFAIK, and if some form of lint gets blessed by incorporation into
> >>> Python, then it needs to be available for all standard installs.
> >> Regardless, the stdlib is for modules to import, tools for programs that
> >>  run.
>
> > We blur that line all the time though (cf. timeit, pdb, pydoc,
> > webbrowser, runpy, probably others).
>
> > So I'd agree with Aahz that if python-dev is going to bless something
> > along these lines, it should be something worthy of inclusion in the
> > standard lib itself rather than just being dropped into the Tools directory.
>
> timit, pdb, webbrowser, amd runpy are importable modules that *also*
> have a command-line interface via "if __name__ == '__main__': ...".
> Pydoc might be also, but I could not be sure from the doc.  If the
> checker were written similarly, so that it could be imported and used
> from within a program to check another file, then I would agree that
> stdlib would be a place for it.
>
> tjr

[snip]

Skipping over some of the wrangling about where it goes for a minute,
is there any firm consensus on which tool to put in? I've only heard
back from pylint, and there seems to be a general feeling that that's
the strictest (and therefore best), but that its external dependencies
constitute a liability. Does anybody have a firm objection on that
score?

Geremy Condra

Ben Finney

unread,
May 6, 2009, 3:14:08 AM5/6/09
to python...@python.org
CTO <deba...@gmail.com> writes:

> Skipping over some of the wrangling about where it goes for a minute,
> is there any firm consensus on which tool to put in? I've only heard
> back from pylint, and there seems to be a general feeling that that's
> the strictest (and therefore best), but that its external dependencies
> constitute a liability. Does anybody have a firm objection on that
> score?

Not a firm objection: pylint is also the most complex (AFAICT). Perhaps
a simpler one that still does a good job would be best for stdlib
consideration.

--
\ “To save the world requires faith and courage: faith in reason, |
`\ and courage to proclaim what reason shows to be true.” |
_o__) —Bertrand Russell |
Ben Finney

Jesse Noller

unread,
May 6, 2009, 8:50:21 AM5/6/09
to Ben Finney, python...@python.org
On Wed, May 6, 2009 at 3:14 AM, Ben Finney <ben+p...@benfinney.id.au> wrote:
> CTO <deba...@gmail.com> writes:
>
>> Skipping over some of the wrangling about where it goes for a minute,
>> is there any firm consensus on which tool to put in? I've only heard
>> back from pylint, and there seems to be a general feeling that that's
>> the strictest (and therefore best), but that its external dependencies
>> constitute a liability. Does anybody have a firm objection on that
>> score?
>
> Not a firm objection: pylint is also the most complex (AFAICT). Perhaps
> a simpler one that still does a good job would be best for stdlib
> consideration.

+1, something which does simple linting, and pep 8 checking would be a
big enough benefit over what we have now. Ideally, it would support
extensibility for additional rules.

jesse

Jeremy Hylton

unread,
May 6, 2009, 2:43:14 PM5/6/09
to Jesse Noller, Ben Finney, python...@python.org
On Wed, May 6, 2009 at 8:50 AM, Jesse Noller <jno...@gmail.com> wrote:
> On Wed, May 6, 2009 at 3:14 AM, Ben Finney <ben+p...@benfinney.id.au> wrote:
>> CTO <deba...@gmail.com> writes:
>>
>>> Skipping over some of the wrangling about where it goes for a minute,
>>> is there any firm consensus on which tool to put in? I've only heard
>>> back from pylint, and there seems to be a general feeling that that's
>>> the strictest (and therefore best), but that its external dependencies
>>> constitute a liability. Does anybody have a firm objection on that
>>> score?
>>
>> Not a firm objection: pylint is also the most complex (AFAICT). Perhaps
>> a simpler one that still does a good job would be best for stdlib
>> consideration.
>
> +1, something which does simple linting, and pep 8 checking would be a
> big enough benefit over what we have now. Ideally, it would support
> extensibility for additional rules.

I don't see much value in building a new lint tool. I might like it
if pylint were less complex than it is, but I don't understand the
design fully enough to have justification for that. If we can get a
good PEP 8 checker out of pylint, it would be silly to write something
from scratch. We went through a similar exercise at work two summers
ago and concluded that we'd it would be faster to build on pylint than
to write something from scratch. I don't think there are any special
considerations for python stdlib development that would lead me to
reach a different conclusion. Certainly, it would be convenient if
the lint tool was part of the source tree, but our editors, compilers,
debuggers, and source control tools aren't part of the source tree
either. I don't think the desire to put it in tools or stdlib
outweighs the cost of building and maintaining a new tool when a
perfectly good one already exists.

Jeremy

CTO

unread,
May 6, 2009, 3:44:46 PM5/6/09
to python...@python.org

On May 6, 2:43 pm, Jeremy Hylton <jer...@alum.mit.edu> wrote:
> On Wed, May 6, 2009 at 8:50 AM, Jesse Noller <jnol...@gmail.com> wrote:

Should we give another look to something like pep8.py, then? Less
complex, certainly, no dependencies and it runs cleanly under python3
after running it through 2to3. On the minus side, it is *much* less
rigorous.

Geremy Condra

CTO

unread,
May 6, 2009, 4:43:23 PM5/6/09
to python...@python.org
> Should we give another look to something like pep8.py, then? Less
> complex, certainly, no dependencies and it runs cleanly under python3
> after running it through 2to3. On the minus side, it is *much* less
> rigorous.
>
> Geremy Condra

I've also just heard back from the maintainer of pyflakes, and he
seems
pretty happy with the idea. Here's the email:

Geremy,

I maintain Pyflakes and am an employee of the company responsible for
its development.

I think it's a good idea; Pyflakes is definitely an important part of
my development process. There are pretty diverse opinions about how
thorough and punctilious Python code checkers should try to be, so
there are a bunch of relatively popular programs with vaguely similar
functionality. I'm a fan of Pyflakes in particular because it's
quick, passive and not overly ambitious or presumptious.

Let me know how things work out.

Take care,
Moe


So, are there any particular objections to pyflakes? Does anybody have
a strong
preference for pylint, pep8.py, or pychecker over pyflakes?

Ben Finney

unread,
May 6, 2009, 7:47:42 PM5/6/09
to python...@python.org
Jeremy Hylton <jer...@alum.mit.edu> writes:

> On Wed, May 6, 2009 at 8:50 AM, Jesse Noller <jno...@gmail.com> wrote:
> > On Wed, May 6, 2009 at 3:14 AM, Ben Finney <ben+p...@benfinney.id.au> wrote:
> >> Not a firm objection: pylint is also the most complex (AFAICT).
> >> Perhaps a simpler one that still does a good job would be best for
> >> stdlib consideration.
> >
> > +1, something which does simple linting, and pep 8 checking would be
> > a big enough benefit over what we have now. Ideally, it would
> > support extensibility for additional rules.
>
> I don't see much value in building a new lint tool. I might like it
> if pylint were less complex than it is, but I don't understand the
> design fully enough to have justification for that. If we can get a
> good PEP 8 checker out of pylint, it would be silly to write something
> from scratch.

AFAIK, nobody is suggesting writing one from scratch; earlier in this
thread we explored the fact that there are several already written and
working.

--
\ “Always code as if the guy who ends up maintaining your code |
`\ will be a violent psychopath who knows where you live.” —John |
_o__) F. Woods |
Ben Finney

Ben Finney

unread,
May 6, 2009, 7:49:31 PM5/6/09
to python...@python.org
CTO <deba...@gmail.com> writes:

> So, are there any particular objections to pyflakes? Does anybody have
> a strong
> preference for pylint, pep8.py, or pychecker over pyflakes?

I'm happy with focussing on pyflakes; it's the one I use most often.

I've tried to love pylint, but configuring it so it's useful per project
was beyond me.

--
\ “If you wish to strive for peace of soul, then believe; if you |
`\ wish to be a devotee of truth, then inquire.” —Friedrich |
_o__) Nietzsche |
Ben Finney

Reply all
Reply to author
Forward
0 new messages