Thoughts on new grammar rules (PEP 284 in particular)
Steve Horne
I was just reading PEP284 and, in general, I like it. The trouble is,
I'd quite like these limit-specifications to be more general. For
example, I'd quite like to write...
if 0 <= x < 10 :
print "x is in range"
There is a big problem with this, of course - this is already legal
Python, but the semantics aren't what I intend.
Looking generally through other PEPs, there seems to be a general
problem with adding new grammar to the Python language. Some
contortions are needed to avoid breaking old code.
I was wondering if maybe it is time that the Python 'core' grammar was
considered broadly complete, with the exception of a special notation
specifically designed for the purpose of wrapping new grammar rules.
I haven't thought through my choice of symbols, but imagine a notation
such as the following was created...
[* indentifier RULE *]
That is, use delimiters with and enclosed identifier and rule where
the identifier explicitly identifies the grammar rule being used.
We could then have...
range checking...
if [* check 0 <= x < 10 *] :
print "OK"
for loop...
for [* range 0 <= x < 10 *] :
pass
Obvious Advantages...
- If the new syntax turns out to be obscure and rarely used, people
reading the code that does use it at least have a clear identifier
name to look up in the manuals to find an explanation of what it
does.
- With careful choice of delimiters (ie the [* and *] above) this
should never cause a code break, and should save a lot of hassle
in this respect in the future.
- Python should be able to give a clear error message if source code
intended for a newer version is used (e.g. 'range support not
provided in this Python version) by simply picking up the
identifier.
Obvious disadvantages...
- Noticably wordier than necessary, as each new grammar rule has
three 'redundant' tokens.
- Could make it too convenient to add obscure special-purpose
features, leading to unnecessary bloat.
Any comments?
--
Steve Horne
st...@lurking.demon.co.uk
David Eppstein
Steve Horne <st...@lurking.demon.co.uk> wrote:
> I was just reading PEP284 and, in general, I like it. The trouble is,
> I'd quite like these limit-specifications to be more general. For
> example, I'd quite like to write...
>
> if 0 <= x < 10 :
> print "x is in range"
>
> There is a big problem with this, of course - this is already legal
> Python, but the semantics aren't what I intend.
I don't understand your point -- what semantics do you intend for this
test that are different from current Python?
--
David Eppstein UC Irvine Dept. of Information & Computer Science
epps...@ics.uci.edu http://www.ics.uci.edu/~eppstein/
holger krekel
>
> I was just reading PEP284 and, in general, I like it. The trouble is,
> I'd quite like these limit-specifications to be more general. For
> example, I'd quite like to write...
>
> if 0 <= x < 10 :
> print "x is in range"
>
> There is a big problem with this, of course - this is already legal
> Python, but the semantics aren't what I intend.
exactly.
> Looking generally through other PEPs, there seems to be a general
> problem with adding new grammar to the Python language. Some
> contortions are needed to avoid breaking old code.
> I was wondering if maybe it is time that the Python 'core' grammar was
> considered broadly complete, with the exception of a special notation
> specifically designed for the purpose of wrapping new grammar rules.
>
> I haven't thought through my choice of symbols, but imagine a notation
> such as the following was created...
>
> [* indentifier RULE *]
ugh :-) Ugly to read...
> if [* check 0 <= x < 10 *] :
> print "OK"
better:
if 0 <= x and x < 10:
> for [* range 0 <= x < 10 *] :
> pass
better:
for range in xrange(0,10):
pass
Tell us the truth! You were sent from these evil perl-hackers
on a quest to bring python to perl's line-noise levels :-)
> Obvious Advantages...
>
> - If the new syntax turns out to be obscure and rarely used, people
> reading the code that does use it at least have a clear identifier
> name to look up in the manuals to find an explanation of what it
> does.
You just made it difficult to read other's code
without looking up some (probably missing:-) documentation.
> - With careful choice of delimiters (ie the [* and *] above) this
> should never cause a code break, and should save a lot of hassle
> in this respect in the future.
while increasing general hassle.
> - Python should be able to give a clear error message if source code
> intended for a newer version is used (e.g. 'range support not
> provided in this Python version) by simply picking up the
> identifier.
I think there is absolutely no chance of something like this ever
beeing integrated.
> - Noticably wordier than necessary, as each new grammar rule has
> three 'redundant' tokens.
> - Could make it too convenient to add obscure special-purpose
> features, leading to unnecessary bloat.
exactly.
> Any comments?
kill that perlish devil in you and enjoy python :-)
regards,
holger
Steve Holden
of the "for" statement this PEP would change. As always, the "if" statement
you quote will work (Python has had chained comparisons since God was a
small girl):
>>> for x in range(-10, 20): # [-10, -9, ... , 19]
... if 0 <= x < 10:
... print x, "is in range"
...
0 is in range
1 is in range
2 is in range
3 is in range
4 is in range
5 is in range
6 is in range
7 is in range
8 is in range
9 is in range
>>>
Hope this makes your weekend brighter!
regards
Steve
--
-----------------------------------------------------------------------
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/pwp/
-----------------------------------------------------------------------
"Steve Horne" <st...@lurking.demon.co.uk> wrote in message
news:82undu4anb47usgck...@4ax.com...
Erik Max Francis
> if 0 <= x < 10 :
> print "x is in range"
>
> There is a big problem with this, of course - this is already legal
> Python, but the semantics aren't what I intend.
This should behave the way you expect. The conditional would be
executed if 0 <= x and x < 10.
--
Erik Max Francis / m...@alcyone.com / http://www.alcyone.com/max/
__ San Jose, CA, US / 37 20 N 121 53 W / ICQ16063900 / &tSftDotIotE
/ \ Who'd ever think it / Such a squalid little ending
\__/ The American and Florence, _Chess_
Church / http://www.alcyone.com/pyos/church/
A lambda calculus explorer in Python.
holger krekel
> I'm not sure what you think would be wrong. It's only the syntax/semantics
> of the "for" statement this PEP would change. As always, the "if" statement
> you quote will work (Python has had chained comparisons since God was a
> small girl):
>
> >>> for x in range(-10, 20): # [-10, -9, ... , 19]
> ... if 0 <= x < 10:
> ... print x, "is in range"
> ...
> 0 is in range
> 1 is in range
> 2 is in range
> 3 is in range
> 4 is in range
> 5 is in range
> 6 is in range
> 7 is in range
> 8 is in range
> 9 is in range
> >>>
david and you are right but additionally the OP suggested
a general [* syntax *] escape mechanism.
holger
Erik Max Francis
> david and you are right but additionally the OP suggested
> a general [* syntax *] escape mechanism.
So what would be the purpose of that if it already behaves the way he
wishes?
Gustavo Niemeyer
>
> range checking...
>
> if [* check 0 <= x < 10 *] :
> print "OK"
>
> for loop...
>
> for [* range 0 <= x < 10 *] :
> pass
Hummm... I'll start to write a museum of weird suggestions.. :-))
--
Gustavo Niemeyer
[ 2AAC 7928 0FBF 0299 5EB5 60E2 2253 B29A 6664 3A0C ]
Terry Reedy
> I was wondering if maybe it is time that the Python 'core' grammar
was
> considered broadly complete,
I've been thinking much the same thing.
>with the exception of a special notation
> specifically designed for the purpose of wrapping new grammar rules.
but to me, complete means "don't make many more changes" rather than
to facilitate making
lots and lots ;<)
Terry J. Reedy
holger krekel
> holger krekel wrote:
>
> > david and you are right but additionally the OP suggested
> > a general [* syntax *] escape mechanism.
>
> So what would be the purpose of that if it already behaves the way he
> wishes?
there was more. he e.g mentioned 'for [* 1<x<10 *]:'
where x would probably iterate from 2 to 9, i guess.
Moreover he was making a general point for syntax-escape mechanisms
(i.e. a parametrized parser, which can be extended in the future
without breaking the syntax and semantics of older programs).
regards,
holger
Alex Martelli
...
> example, I'd quite like to write...
>
> if 0 <= x < 10 :
> print "x is in range"
>
> There is a big problem with this, of course - this is already legal
> Python, but the semantics aren't what I intend.
As many have pointed out, this is probably not true.
> I haven't thought through my choice of symbols, but imagine a notation
> such as the following was created...
>
> [* indentifier RULE *]
One stray thought I had at IPC10 after listening to some of Tim
Berners-Lee's wildest syntax suggestion was to allow some "compile
time function call" notation such as...:
identifier(:whatever:)
This would basically be the same as:
identifier('''whatever''')
except that the compiler would notice the (: ... :) form and perform
the call *during compilation* -- the function would probably have
to be defined with a similar "compile-time function definition":
def identifier(:formalarg:):
...
notation, and would return any object suitable as first argument
to builtin function 'compile' (codeobject, string) so code generation
could proceed.
Guido started quizzing me when I mentioned it and it rapidly
emerged that I hadn't thought it through -- as indeed I hadn't,
it WAS just a stray thought. He even mentioned that this could be
seen as a 'macro', thereby causing me to recoil in horror (I've
seen what macros do to languages and don't like it one bit:-).
> - Noticably wordier than necessary, as each new grammar rule has
> three 'redundant' tokens.
Nah...:
loop_with(: 1 <= x <= 10 :) :
print x
has fewer tokens than today's "for x in range(1, 11):" equivalent.
That's definitely not the problem.
> - Could make it too convenient to add obscure special-purpose
> features, leading to unnecessary bloat.
Ay, there's the rub. Multiple mutually incompatible bloatings
as each group of clever people rushes off to invent their own
clever sets of macros (<horror> <recoil/> </horror>).
Alex
David LeBlanc
> From: python-l...@python.org
> [mailto:python-l...@python.org]On Behalf Of Alex Martelli
> Sent: Saturday, May 11, 2002 2:53
> To: pytho...@python.org
> Subject: Re: Thoughts on new grammar rules (PEP 284 in particular)
<snip>
> > - Could make it too convenient to add obscure special-purpose
> > features, leading to unnecessary bloat.
>
> Ay, there's the rub. Multiple mutually incompatible bloatings
> as each group of clever people rushes off to invent their own
> clever sets of macros (<horror> <recoil/> </horror>).
>
>
> Alex
Then the bloatings harden and you have a perl ;-)
I just read GvR's comment on this PEP on python.org, and I agree with his
observation that having the controlling iteration variable buried amid the
expression could be confusing.
Dave LeBlanc
Seattle, WA USA
Steve Horne
<epps...@ics.uci.edu> wrote:
>In article <82undu4anb47usgck...@4ax.com>,
> Steve Horne <st...@lurking.demon.co.uk> wrote:
>
>> I was just reading PEP284 and, in general, I like it. The trouble is,
>> I'd quite like these limit-specifications to be more general. For
>> example, I'd quite like to write...
>>
>> if 0 <= x < 10 :
>> print "x is in range"
>>
>> There is a big problem with this, of course - this is already legal
>> Python, but the semantics aren't what I intend.
>
>I don't understand your point -- what semantics do you intend for this
>test that are different from current Python?
That was actually a mistake - I use C++ more than Python and stupidly
did not test this before assuming C-like semantics.
In C...
x = 15
0 <= x < 10
= (0 <= x) < 10
= 1 < 10
= 1
which is not what is intended. Note - I did not check the
associativity of these operators, but the principle is correct - the
result would not be a two-ended range check.
Unfortunately for my argument, Python does 'the right thing' already.
--
Steve Horne
st...@lurking.demon.co.uk
Steve Horne
<py...@devel.trillke.net> wrote:
>Tell us the truth! You were sent from these evil perl-hackers
>on a quest to bring python to perl's line-noise levels :-)
Nope - I hate perl.
The basis is taken from XML (or, alternately, LISP).
In a manual, it is much easier to look up a name than a combination of
operators. For instance, on the first reading of Python code, if you
did not know that the '{', ':', and '}' tokens combined to define
dictionaries, it would be a bit of a pain to look up in the manual
while trying to read some source code. You wouldn't know to look up
'dictionary' because you wouldn't know it's a dictionary.
For such a fundamental feature as dictionaries the answer is 'read the
beginners tutorial', but with more obscure features an answer of 'read
the entire Python documentation' is unreasonable to say the least.
My idea is not practical for everyday features, and I'm not suggesting
it for that. The Python language is already more than rich enough to
support most everyday programming tasks easily. But it allows new
features to be added, and it's no less readable than an calculation
with a few redundant parentheses surely. As such, surely it's better
than performing ever-more-extreme contortions with the existing
keywords and operators, as is already noticable in some PEPs.
Of course, with 20/20 hindsight, a much better approach could have
been taken. For instance, consider the following...
begin function ... :
...
begin while ... :
...
And, relevant to a current PEP...
begin switch ... :
begin case ... :
...
begin default :
...
and because only 'begin' would be a keyword ('procedure' etc being
ordinary identifiers, which only have special meaning after the
'begin' keyword) any new structure could be added without needing new
keywords or obscure operator combinations - old code would be safe
from at least one aspect of the evolution of the language.
By using 'begin' to make the start of a structure explicit, the ':'
could also probably be dropped. I'm always forgetting them things
(whinge whine...).
So using perlish symbols isn't by any means my ideal - I just think
that in the long run there may already be a risk of serious creaping
perlishness (due to the resistance to adding new keywords), and this
would be a way of saying 'this far, but no further'.
--
Steve Horne
st...@lurking.demon.co.uk
Steve Horne
wrote:
>He even mentioned that this could be
>seen as a 'macro', thereby causing me to recoil in horror (I've
>seen what macros do to languages and don't like it one bit:-).
I certainly wasn't suggesting macros. The new facilities would still
need to be accepted and built into the interpreter with my scheme -
and the main bloat filter (objecting Python users) would still be
present.
Actually, though, I rather like macros - just not the way they are
typically handled. Sometimes, code generation can be a better approach
than hand coding. As with any feature, of course, it can be abused -
but that's beside the point. After all, even the unary '-' can be
abused (what is the value of -----------------------------------1 ?)
Ideally, I'd like compiled languages to support this directly - with
the features being invoked at compile (or code generation) time being
as-far-as-possible identical (even in syntax) to those available in
the main code. In fact, I see this as an extreme case of the
optimisation view that if something has no dependency on run-time
data, it should be evaluated at compile-time.
ASP and the like have already approached that problem - providing
'escapes' to say a section of a document is 'calculated' by the server
prior to sending it to the clients browser. I have already written a
code-generation tool which detects some simple escapes while
preprocessing Python source code, so I can use Python (with a few
minor tweaks) to generate C++. It works well enough - but as with any
system of escapes, if overused, it can make a mess.
--
Steve Horne
st...@lurking.demon.co.uk
holger krekel
> On Fri, 10 May 2002 19:32:17 +0200, holger krekel
> <py...@devel.trillke.net> wrote:
>
> >Tell us the truth! You were sent from these evil perl-hackers
> >on a quest to bring python to perl's line-noise levels :-)
>
> Nope - I hate perl.
>
> The basis is taken from XML (or, alternately, LISP).
>
> In a manual, it is much easier to look up a name than a combination of
> operators. For instance, on the first reading of Python code, if you
> did not know that the '{', ':', and '}' tokens combined to define
> dictionaries, it would be a bit of a pain to look up in the manual
> while trying to read some source code. You wouldn't know to look up
> 'dictionary' because you wouldn't know it's a dictionary.
mostly true. But David Beazley's Essential Reference does it right:
it lists many symbols at the beginning of the index (except ':').
> My idea is not practical for everyday features, and I'm not suggesting
> it for that. The Python language is already more than rich enough to
> support most everyday programming tasks easily. But it allows new
> features to be added, and it's no less readable than an calculation
> with a few redundant parentheses surely. As such, surely it's better
> than performing ever-more-extreme contortions with the existing
> keywords and operators, as is already noticable in some PEPs.
>
> Of course, with 20/20 hindsight, a much better approach could have
> been taken. For instance, consider the following...
>
> begin function ... :
> ...
>
> begin while ... :
> ...
> And, relevant to a current PEP...
>
> begin switch ... :
> begin case ... :
> ...
>
> begin default :
> ...
don't you think that the increased verbosity hurts a lot?
(Apart from the fact that 'begin' sounds like starting a loop to me.)
> and because only 'begin' would be a keyword ('procedure' etc being
> ordinary identifiers, which only have special meaning after the
> 'begin' keyword) any new structure could be added without needing new
> keywords or obscure operator combinations - old code would be safe
> from at least one aspect of the evolution of the language.
not true, unless the parser doesn't look at your code-block. Consider
the 'yield' keyword. Older python-parsers produce a 'SyntaxError'.
If you would esacape that with 'begin yield' then the parsing process
starts to loose a lot of meaning (apart from beeing very ugly).
> So using perlish symbols isn't by any means my ideal - I just think
> that in the long run there may already be a risk of serious creaping
> perlishness (due to the resistance to adding new keywords)
um, this sentence somewhat means that python is already perlish :-)
> , and this
> would be a way of saying 'this far, but no further'.
IMO the developer crew and especially GvR is very careful
with respect to language changes. I don't know if
it would be possible to be even more conservative about
language *syntax* so that older parsers would not choke
on future extensions.
holger
Steve Horne
<py...@devel.trillke.net> wrote:
>> Of course, with 20/20 hindsight, a much better approach could have
>> been taken. For instance, consider the following...
>>
>> begin function ... :
>> ...
>>
>> begin while ... :
>> ...
>> And, relevant to a current PEP...
>>
>> begin switch ... :
>> begin case ... :
>> ...
>>
>> begin default :
>> ...
>
>don't you think that the increased verbosity hurts a lot?
>(Apart from the fact that 'begin' sounds like starting a loop to me.)
Not really, with just this one keyword. If you've ever used Ada,
you'll have seen much more verbosity than this. In fact, most of the
block structures in Pascal, Modula 2, Ada and similar require a begin
keyword - but it is positioned in such a way that it can't be used as
a 'here is a block structure' hint, so multiple predefined keywords
are needed in each case.
Avoiding verbosity is important, but it always involves trade-offs
with other language goals. Python is already frequently more verbose
than Perl - for example we need a 'len' function rather than a 'scalar
context' symbol to determine the length of a list - because
readability is given a higher priority in Python. I simply suspect
that it might be beneficial to consider a similar trade-off for
grammar-extensibility.
As for the appearance of starting a loop, why? - it is already used in
a number of languages as part of the start of structure (in Pascal
particularly, playing much the same role as the C '{' marker). In Ada,
loops are among the structures it is *not* used for. To my knowledge,
users of these languages don't suffer from confusion on this issue.
>> and because only 'begin' would be a keyword ('procedure' etc being
>> ordinary identifiers, which only have special meaning after the
>> 'begin' keyword) any new structure could be added without needing new
>> keywords or obscure operator combinations - old code would be safe
>> from at least one aspect of the evolution of the language.
>
>not true, unless the parser doesn't look at your code-block. Consider
>the 'yield' keyword. Older python-parsers produce a 'SyntaxError'.
>If you would esacape that with 'begin yield' then the parsing process
>starts to loose a lot of meaning (apart from beeing very ugly).
By 'one aspect', I mean the identification of block structures, not
any special statements used inside the blocks. 'break', 'continue',
'return' and the like would need another mechanism to support
extensions without new keywords, if that was considered worthwhile.
However, in this alternative universe, there would be no need for
yield. The more sensible (IMHO) alternative would be to start
generators using 'begin generator (args)' and use the return statement
to return values from the generator.
>> So using perlish symbols isn't by any means my ideal - I just think
>> that in the long run there may already be a risk of serious creaping
>> perlishness (due to the resistance to adding new keywords)
>
>um, this sentence somewhat means that python is already perlish :-)
Yes. For instance, to determine whether a 'def' definition is for a
function or a generator, you cannot simply look at its definition -
you have to search the body to see if it contains 'yield' statements
or not and infer whether it is a generator from that. It is not
explicit from the start what you are dealing with, and it is therefore
a bit unclear and potentially misleading and confusing.
Seems a tad perlish to me :-(
Please note - this is not intended as a serious criticism. Generators
are a valuable addition to the language and, given the pre-existing
circumstances, this syntax had as much merit as any other. I'm simply
suggesting that, in an alternative universe where the language grammar
was designed to be easily extensible from the start, language
extensions such as generators might have been more easily and clearly
accomodated without creating compatibility issues.
>IMO the developer crew and especially GvR is very careful
>with respect to language changes. I don't know if
>it would be possible to be even more conservative about
>language *syntax* so that older parsers would not choke
>on future extensions.
I agree. Very much so. I just think that there might be a better path
than going through ever-increasing contortions to maintain
compatibility when extending the language.
--
Steve Horne
st...@lurking.demon.co.uk