[Python-ideas] Statements vs Expressions... why?
Cliff Wells
Something that has started to annoy me in the last couple of years is
the fact that most Python control statements cannot be used as
expressions. I feel this is a pretty deep limitation and personally I
don't feel it's well-justified.
As I understand it, the reason for the distinction mostly has to do with
the premise "flat is better than nested", which I can understand, but I
don't think carries enough weight anymore.
Specifically, I'd like to see things like "if" statements, "for" loops,
etc, become expressions. This would not preclude them from being used
as if they were statements (an expression can stand alone on a line of
Python code), but would add a lot of expressiveness to the language as
well as make Python more viable for creating DSLs.
Additionally, removing statements from Python would also allow the
language to be simplified. No need for a ternary "if" operator with
different semantics than a normal "if" statement, "for" loops would be
brought closer to generators in functionality, and the perceived
limitations of lambda would disappear, amongst other things. We'd gain
a lot of features found in languages like Lisp and Ruby as a side-effect
(i.e. anonymous code blocks).
Overall it seems this design decision is specifically geared toward
forcing programmers into an imperative style in order to enforce program
readability. In Python 1.5, this made a bit of sense, but as Python has
"matured" (or in my view, gotten over-complicated) this makes much less
sense. Many parts of Python's extensive syntax are explicit workarounds
to this design decision. So on the one hand we have the perceived
danger that programmers will write nested code and on the other we have
an ever-expanding syntax. I'd take the former any day.
I've not delved into the internals of the Python interpreter to check,
but I suspect that converting most statements to expressions would not
be very difficult (changing the grammar definition and generated
bytecode a small amount in most cases).
Any thoughts on this? I'm sure it's been brought up before, but I
haven't found any definitive discussions on why this rather arbitrary
design decision continues to hold in the face of a general migration
away from imperative languages (especially when it seems it could be
changed without much backwards-compatibility issues).
Regards,
Cliff
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas
Christian Heimes
> Any thoughts on this? I'm sure it's been brought up before, but I
> haven't found any definitive discussions on why this rather arbitrary
> design decision continues to hold in the face of a general migration
> away from imperative languages (especially when it seems it could be
> changed without much backwards-compatibility issues).
Two thoughts:
Please elaborate how you like to change the syntax of Python. I like to
see some concrete examples how your syntax would look like. I also like
to know how your are planing to implement features like lazy evaluation.
The if else ternary operator statement is evaluated lazy. The same
construct as expression wouldn't be lazy any more.
Secondly any syntax change won't happen until we start planing Python
4000 ;)
Christian
Blake Winton
> I'd like to see things like "if" statements, "for" loops,
> etc, become expressions.
> Any thoughts on this? I'm sure it's been brought up before, but I
> haven't found any definitive discussions on why this rather arbitrary
> design decision continues to hold in the face of a general migration
> away from imperative languages (especially when it seems it could be
> changed without much backwards-compatibility issues).
Can you show me what an "if" in a "lambda" used in a function call would
look like? My major complaint with the statements-as-expressions is
that multi-line statements really don't look good when used in
expression contexts. Perhaps you have a good suggestion for the syntax,
and if so, I'ld love to see it.
> I've not delved into the internals of the Python interpreter to check,
> but I suspect that converting most statements to expressions would not
> be very difficult (changing the grammar definition and generated
> bytecode a small amount in most cases).
Well, I suspect that delving into the internals to check this theory
would make people take your proposal a lot more seriously. Heck, I'ld
go so far as to say that without doing that, it's all just talk, and
talk is cheap. ;)
Thanks,
Blake.
Arnaud Delobelle
[Sorry for the private reply earlier]
On 10 Sep 2008, at 19:43, Cliff Wells wrote:
> Greetings,
>
> Something that has started to annoy me in the last couple of years is
> the fact that most Python control statements cannot be used as
> expressions. I feel this is a pretty deep limitation and personally I
> don't feel it's well-justified.
>
> As I understand it, the reason for the distinction mostly has to do
> with
> the premise "flat is better than nested", which I can understand,
> but I
> don't think carries enough weight anymore.
>
> Specifically, I'd like to see things like "if" statements, "for"
> loops,
> etc, become expressions. This would not preclude them from being used
> as if they were statements (an expression can stand alone on a line of
> Python code), but would add a lot of expressiveness to the language as
> well as make Python more viable for creating DSLs.
>
Can you post some sample code to illustrate how statements could be
used as expressions?
Do you propose that we write:
y = if x == 0:
0
else:
exp(x**-2)
instead of:
y = 0 if x == 0 else exp(x**-2)
? Or, how would you write
factors = [x for x in range(2, n) if n % x == 0]
? Something like this maybe:
factors = for x in range(2, n):
if n % x == 0:
x
Or do you suggest something else?
> Additionally, removing statements from Python would also allow the
> language to be simplified. No need for a ternary "if" operator with
> different semantics than a normal "if" statement, "for" loops would be
> brought closer to generators in functionality, and the perceived
> limitations of lambda would disappear, amongst other things. We'd
> gain
> a lot of features found in languages like Lisp and Ruby as a side-
> effect
> (i.e. anonymous code blocks).
>
> Overall it seems this design decision is specifically geared toward
> forcing programmers into an imperative style in order to enforce
> program
> readability. In Python 1.5, this made a bit of sense, but as Python
> has
> "matured" (or in my view, gotten over-complicated) this makes much
> less
> sense. Many parts of Python's extensive syntax are explicit
> workarounds
> to this design decision. So on the one hand we have the perceived
> danger that programmers will write nested code and on the other we
> have
> an ever-expanding syntax. I'd take the former any day.
>
So do you think readability is not as important now as it was?
> I've not delved into the internals of the Python interpreter to check,
> but I suspect that converting most statements to expressions would not
> be very difficult (changing the grammar definition and generated
> bytecode a small amount in most cases).
>
> Any thoughts on this? I'm sure it's been brought up before, but I
> haven't found any definitive discussions on why this rather arbitrary
> design decision continues to hold in the face of a general migration
> away from imperative languages (especially when it seems it could be
> changed without much backwards-compatibility issues).
I think to call this feature of Python an arbitrary design decision is
a misjudgement. To me it is central to the identity of Python.
--
Arnaud
Cliff Wells
Yes, and parentheses could be used to disambiguate, as anywhere else.
> ? Or, how would you write
>
> factors = [x for x in range(2, n) if n % x == 0]
>
> ? Something like this maybe:
>
> factors = for x in range(2, n):
> if n % x == 0:
> x
>
> Or do you suggest something else?
These are correct, albeit simple examples (you are using only assignment
examples). To give something more interesting, I'd like to be able to
do things like this:
dispatch = {
'1': lambda x: (
for i in range(x):
if not x % 2:
yield 0
else:
yield 1
),
'2': lambda x: (
for i in range(x):
yield i
)
}
for i in dispatch[val](1):
print i
Note that this isn't just about lambda, but rather general
expressiveness.
To clarify, I propose *not* making syntax changes that add to the
language or alter the meaning of existing code.
Rather my proposal is along the lines of "relax this particular syntax
requirement" and let the chips fall where they may. In other words,
Python 2.x code would still work, it simply would not take advantage of
a particular coding style now available to it.
The key difference between a statement and an expression is that
1) an expression has a return value, a statement does not
2) statements cannot be used inside of expressions
In short, statements are more-or-less castrated expressions. They have
no other special features.
> > Additionally, removing statements from Python would also allow the
> > language to be simplified. No need for a ternary "if" operator with
> > different semantics than a normal "if" statement, "for" loops would be
> > brought closer to generators in functionality, and the perceived
> > limitations of lambda would disappear, amongst other things. We'd
> > gain
> > a lot of features found in languages like Lisp and Ruby as a side-
> > effect
> > (i.e. anonymous code blocks).
> >
> > Overall it seems this design decision is specifically geared toward
> > forcing programmers into an imperative style in order to enforce
> > program
> > readability. In Python 1.5, this made a bit of sense, but as Python
> > has
> > "matured" (or in my view, gotten over-complicated) this makes much
> > less
> > sense. Many parts of Python's extensive syntax are explicit
> > workarounds
> > to this design decision. So on the one hand we have the perceived
> > danger that programmers will write nested code and on the other we
> > have
> > an ever-expanding syntax. I'd take the former any day.
> >
>
> So do you think readability is not as important now as it was?
Of course not. I'm saying that due to this limitation, readability is
going down the drain even faster. It's getting to the point where it's
not possible to keep the whole of Python in your head and I feel that a
significant portion of this is due to this particular inflexibility.
Further, I feel that this limitation forces programmers into using hacks
and magic or overly spread-out code, which itself leads to readability
concerns. Having used Python for around a decade, I'm quite aware of
the fact that you can happily write tons and tons of nice code with
Python in its current state. However, because of the direction of my
work (developing a internal DSL in Python) I've suddenly become aware of
this glass ceiling. I'd bumped into it before back when I was doing a
lot of GUI development, but blamed it on lambda rather than realizing
that it wasn't lambda so much as what I am bringing up now.
> > I've not delved into the internals of the Python interpreter to check,
> > but I suspect that converting most statements to expressions would not
> > be very difficult (changing the grammar definition and generated
> > bytecode a small amount in most cases).
> >
> > Any thoughts on this? I'm sure it's been brought up before, but I
> > haven't found any definitive discussions on why this rather arbitrary
> > design decision continues to hold in the face of a general migration
> > away from imperative languages (especially when it seems it could be
> > changed without much backwards-compatibility issues).
>
> I think to call this feature of Python an arbitrary design decision is
> a misjudgement. To me it is central to the identity of Python.
Sorry, I misspoke: it's not an arbitrary *decision*, but it is an
arbitrary *distinction*.
As I said, I'm aware of why this decision was originally made
(prevention of nested code, a.k.a. fear of lisp), but the distinction
itself is completely artificial, created to enforce a model of
programming, rather than for technical or performance reasons.
I agree that it is a distinguishing feature of Python, but I don't think
it is central to Python's identity. That is, it wouldn't be "not
Python" were it removed.
Regards,
Cliff
Cliff Wells
> Cliff Wells wrote:
> > Any thoughts on this? I'm sure it's been brought up before, but I
> > haven't found any definitive discussions on why this rather arbitrary
> > design decision continues to hold in the face of a general migration
> > away from imperative languages (especially when it seems it could be
> > changed without much backwards-compatibility issues).
>
> Two thoughts:
>
> Please elaborate how you like to change the syntax of Python.
No changes. Simply lifting of a particular restriction.
> I like to
> see some concrete examples how your syntax would look like. I also like
> to know how your are planing to implement features like lazy evaluation.
> The if else ternary operator statement is evaluated lazy. The same
> construct as expression wouldn't be lazy any more.
Why not?
a = (
if a > 1 then:
long_calculation()
else:
other_long_calculation()
)
Clearly only one of these blocks would be evaluated at runtime.
>
> Secondly any syntax change won't happen until we start planing Python
> 4000 ;)
Yes, that's my expectation, although hopefully PyPy will make some of
these things possible to experiment with well before then =)
Cliff
Leonardo Santagada
On Sep 10, 2008, at 6:22 PM, Cliff Wells wrote:
> On Wed, 2008-09-10 at 21:46 +0200, Christian Heimes wrote:
>> Cliff Wells wrote:
>>> Any thoughts on this? I'm sure it's been brought up before, but I
>>> haven't found any definitive discussions on why this rather
>>> arbitrary
>>> design decision continues to hold in the face of a general migration
>>> away from imperative languages (especially when it seems it could be
>>> changed without much backwards-compatibility issues).
>>
>> Two thoughts:
>>
>> Please elaborate how you like to change the syntax of Python.
>
> No changes. Simply lifting of a particular restriction.
Two restrictions, both that statements can be used in place of
expressions and that statements now return values. But please explain
how to do it in a way that is clear. It's a plus if it is backwards
compatible :)
>> I like to
>> see some concrete examples how your syntax would look like. I also
>> like
>> to know how your are planing to implement features like lazy
>> evaluation.
>> The if else ternary operator statement is evaluated lazy. The same
>> construct as expression wouldn't be lazy any more.
>
> Why not?
>
> a = (
> if a > 1 then:
> long_calculation()
> else:
> other_long_calculation()
> )
>
> Clearly only one of these blocks would be evaluated at runtime.
And now ifs return a value
>>
>> Secondly any syntax change won't happen until we start planing Python
>> 4000 ;)
>
> Yes, that's my expectation, although hopefully PyPy will make some of
> these things possible to experiment with well before then =)
You can do that now in PyPy or any other python version... or maybe
use Logix to show us a proof of concept version: http://www.livelogix.net/logix/
Now if I remeber correctly this is already the case of logix, look at:
http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html#3.1
Why would you need lambda then? couldn't you just write x = def (x,y):
return x+y ?
Cliff Wells
> Specifically, I'd like to see things like "if" statements, "for" loops,
> etc, become expressions. This would not preclude them from being used
> as if they were statements (an expression can stand alone on a line of
> Python code), but would add a lot of expressiveness to the language as
> well as make Python more viable for creating DSLs.
Because I know that the question "what's your use case" will inevitably
be forwarded, I will explain the exact use case that has driven me to
distraction over this limitation.
I developed and maintain a template engine written in Python:
http://breve.twisty-industries.com/
As you can see from the example on the main page, Breve is written as an
internal DSL (that is, it compiles directly to Python bytecode... Breve
templates *are* Python). This means that any limitation in Python is
inherently a limitation in Breve.
There were several design decisions to be made when designing Breve, the
first and foremost being "will Breve templates be full-blown Python
programs or will they be limited to expressions?". I chose the latter
for a few reasons:
1) Python expressions map more directly to the generated XML/HTML
output. In other words, a Breve template has the same structure (if not
syntax) as the desired result.
2) It would have required additional syntax within the template to allow
for all of Python to be supported.
Overall, I'm happy with my choice, but it forces some mental twister
when trying to do particular operations in Breve, notably
presentation-logic and looping. A loop must be presented as a list
comprehension. "If" logic must be done using either 2.5's ternary if
operator or using short-circuit logical operations, i.e. "test(value)
and (this)". At one point I actually implemented custom tags for doing
logical operations, but due to the inability to provide
short-circuiting, they were less than satisfactory.
This isn't fatal to Breve (albeit annoying), but it made me painfully
aware of this self-imposed limitation in Python. It also made me wonder
what other concepts this limitation has prevented me from grokking over
this last decade. It's generally accepted that a language constrains
not only the solutions to a problem, but also the *possible* solutions
that a programmer will see. This is as much of a concern to me as any
practical concerns today.
Adam Olsen
> Further, I feel that this limitation forces programmers into using hacks
> and magic or overly spread-out code, which itself leads to readability
> concerns. Having used Python for around a decade, I'm quite aware of
> the fact that you can happily write tons and tons of nice code with
> Python in its current state. However, because of the direction of my
> work (developing a internal DSL in Python) I've suddenly become aware of
> this glass ceiling. I'd bumped into it before back when I was doing a
> lot of GUI development, but blamed it on lambda rather than realizing
> that it wasn't lambda so much as what I am bringing up now.
Python is not intended for DSLs. Really, don't do it. Python is for
python code. If you want another language, write your own parser. I
hear lisp is simple to parse, and has no annoying statements to hold
you back!
Seriously though, there is an advantage to basing so much on
statements rather than expressions. We're specialized for one
statement per line, which is the most common case, and it allowed us
to have extraneous semicolons, braces, or whatever. Readability
benefits, style consistency benefits.
Now there are some use cases that suffer here, including the one you
just gave: defining a dispatch dict with the functions inline. The
best I could do is define the dict first, then stick a decorator on
each function to register them. That's still ugly though. A creative
solution is needed, but none come to mind.
An example where this has happened before is the with-statement, which
is spectacularly successful IMO. Now, you may notice it could have
been done in a library rather than core syntax if generic anonymous
blocks were allowed — so what? The library is still part of the
language! It's still something that has to be learned. And the
syntax would be substantially uglier using a generic mechanism, rather
than the specialized with-statement syntax.
--
Adam Olsen, aka Rhamphoryncus
Cliff Wells
> On Sep 10, 2008, at 6:22 PM, Cliff Wells wrote:
>
> > On Wed, 2008-09-10 at 21:46 +0200, Christian Heimes wrote:
> >> Cliff Wells wrote:
> >>> Any thoughts on this? I'm sure it's been brought up before, but I
> >>> haven't found any definitive discussions on why this rather
> >>> arbitrary
> >>> design decision continues to hold in the face of a general migration
> >>> away from imperative languages (especially when it seems it could be
> >>> changed without much backwards-compatibility issues).
> >>
> >> Two thoughts:
> >>
> >> Please elaborate how you like to change the syntax of Python.
> >
> > No changes. Simply lifting of a particular restriction.
>
> Two restrictions, both that statements can be used in place of
> expressions and that statements now return values. But please explain
> how to do it in a way that is clear. It's a plus if it is backwards
> compatible :)
Well, one or two, depending on how you word it ;-) My position is to
simply do away with statements and provide equivalent expressions.
As far as backwards-compatibility, this is legal Python:
1
2
3
It doesn't do anything, but it is valid syntax. This demonstrates that
expressions can be valid when used independent of a statement,
therefore, the following is also legal (assuming "if" were now
implemented as an expression):
if (a==1):
pass
> >> I like to
> >> see some concrete examples how your syntax would look like. I also
> >> like
> >> to know how your are planing to implement features like lazy
> >> evaluation.
> >> The if else ternary operator statement is evaluated lazy. The same
> >> construct as expression wouldn't be lazy any more.
> >
> > Why not?
> >
> > a = (
> > if a > 1 then:
> > long_calculation()
> > else:
> > other_long_calculation()
> > )
> >
> > Clearly only one of these blocks would be evaluated at runtime.
>
> And now ifs return a value
Which you are not required to use, just as today:
def foo(x):
return x**2
foo(2)
>
> >>
> >> Secondly any syntax change won't happen until we start planing Python
> >> 4000 ;)
> >
> > Yes, that's my expectation, although hopefully PyPy will make some of
> > these things possible to experiment with well before then =)
>
> You can do that now in PyPy or any other python version... or maybe
> use Logix to show us a proof of concept version: http://www.livelogix.net/logix/
>
> Now if I remeber correctly this is already the case of logix, look at:
> http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html#3.1
Yes, Logix is quite interesting. Unfortunately the author has
discontinued work on it (and was, in fact, unreachable when I last tried
to contact him). He also mentioned retargeting to another VM (rather
than the Python VM) which made it less appealing to me.
It is interesting that you mention Logix, since it demonstrates that
it's quite possible to be backward-compatible with this particular
change.
> Why would you need lambda then? couldn't you just write x = def (x,y):
> return x+y ?
Yes, although I suspect that this particular change to "def" would be
more substantial and "lambda" is currently equivalent. Although this
might satisfy GvR's inclination to dump lambda at some future point ;-)
Cliff
Cliff Wells
> On Wed, Sep 10, 2008 at 3:18 PM, Cliff Wells <cl...@develix.com> wrote:
> > Further, I feel that this limitation forces programmers into using hacks
> > and magic or overly spread-out code, which itself leads to readability
> > concerns. Having used Python for around a decade, I'm quite aware of
> > the fact that you can happily write tons and tons of nice code with
> > Python in its current state. However, because of the direction of my
> > work (developing a internal DSL in Python) I've suddenly become aware of
> > this glass ceiling. I'd bumped into it before back when I was doing a
> > lot of GUI development, but blamed it on lambda rather than realizing
> > that it wasn't lambda so much as what I am bringing up now.
>
> Python is not intended for DSLs. Really, don't do it. Python is for
> python code.
The DSL I work on *is* Python code. And really, this is the first time
I've heard anyone assert anything like this. Python is a
general-purpose language. It's not VBA ;-)
DSL's are an extremely useful concept. To summarily dispatch the whole
of them with such an assertion is pretty much bolstering my argument:
you've just asserted that Python is inherently limited in scope.
> If you want another language, write your own parser. I
> hear lisp is simple to parse, and has no annoying statements to hold
> you back!
Ah, except Python is the language I like in every way, *except* for this
one particular wart. Really, had I not entered new programming domains
and tried to take Python with me, I'd probably never have had a
complaint.
Also, external parsers defeat the entire reasoning behind internal DSL's
(taking advantage of an established VM/compiler, requiring users to
learn a new syntax in addition to their primary programming language).
> Seriously though, there is an advantage to basing so much on
> statements rather than expressions. We're specialized for one
> statement per line, which is the most common case,
Clearly it's the most common case in existing Python code since nothing
else is allowed. But frankly, even Javascript doesn't follow this
idiom anymore. Expression-oriented languages have seen a rebirth for
good reasons (although I admit I'm none-to-fond of many of them, for
various reasons).
> and it allowed us
> to have extraneous semicolons, braces, or whatever.
Not following this. You mean to *not* have extraneous syntax?
> Readability benefits, style consistency benefits.
I strongly disagree. The artificial distinction between statements and
expressions is the definition of *inconsistent*. Why do we have two
versions of the "if" conditional? Why do we have "for" loops *and* list
comprehensions? They express the same ideas, but the limitations in one
required growing the language another direction. In short, we could
have had a single, more powerful construct in place of two lesser
constructs and simultaneously had less syntax to memorize and more
consistency across the board.
> Now there are some use cases that suffer here, including the one you
> just gave: defining a dispatch dict with the functions inline. The
> best I could do is define the dict first, then stick a decorator on
> each function to register them. That's still ugly though. A creative
> solution is needed, but none come to mind.
That's because there is none. And this is my fundamental problem: it's
not so much that it's hard to do in Python, it's that you *cannot* do it
in Python. No amount of creativity, time, or experience will help, and
this is disappointing.
I won't pretend that any example we might toss up in here won't appear
contrived, but there are definite cases where readability can be
substantially enhanced with such structures.
> An example where this has happened before is the with-statement, which
> is spectacularly successful IMO. Now, you may notice it could have
> been done in a library rather than core syntax if generic anonymous
> blocks were allowed — so what? The library is still part of the
> language! It's still something that has to be learned. And the
> syntax would be substantially uglier using a generic mechanism, rather
> than the specialized with-statement syntax.
The "so what" is that it could *only* be implemented by the core devs.
It was not possible for an average (or even above-average) Python
programmer to write such a library, whereas it *could* have been had the
language not prohibited it.
Regards,
Cliff
Adam Olsen
> On Wed, 2008-09-10 at 15:57 -0600, Adam Olsen wrote:
>> On Wed, Sep 10, 2008 at 3:18 PM, Cliff Wells <cl...@develix.com> wrote:
>> > Further, I feel that this limitation forces programmers into using hacks
>> > and magic or overly spread-out code, which itself leads to readability
>> > concerns. Having used Python for around a decade, I'm quite aware of
>> > the fact that you can happily write tons and tons of nice code with
>> > Python in its current state. However, because of the direction of my
>> > work (developing a internal DSL in Python) I've suddenly become aware of
>> > this glass ceiling. I'd bumped into it before back when I was doing a
>> > lot of GUI development, but blamed it on lambda rather than realizing
>> > that it wasn't lambda so much as what I am bringing up now.
>>
>> Python is not intended for DSLs. Really, don't do it. Python is for
>> python code.
>
> The DSL I work on *is* Python code. And really, this is the first time
> I've heard anyone assert anything like this. Python is a
> general-purpose language. It's not VBA ;-)
>
> DSL's are an extremely useful concept. To summarily dispatch the whole
> of them with such an assertion is pretty much bolstering my argument:
> you've just asserted that Python is inherently limited in scope.
I agree, DSL's are useful. They're just not something python
currently supports well. They're the use-case you need to justify
against the substantial changes you propose. Maybe it's worth it, or
maybe it's better to add an indent-sensitive string literal that would
then allow *arbitrary* DSL syntax?
>> If you want another language, write your own parser. I
>> hear lisp is simple to parse, and has no annoying statements to hold
>> you back!
>
> Ah, except Python is the language I like in every way, *except* for this
> one particular wart. Really, had I not entered new programming domains
> and tried to take Python with me, I'd probably never have had a
> complaint.
>
> Also, external parsers defeat the entire reasoning behind internal DSL's
> (taking advantage of an established VM/compiler, requiring users to
> learn a new syntax in addition to their primary programming language).
I appreciate what you're saying here, and feel much the same way about
my own pet-features, but this is a really poor argument. Everybody
has just one wafer-thin feature they'd like to add.
>> Seriously though, there is an advantage to basing so much on
>> statements rather than expressions. We're specialized for one
>> statement per line, which is the most common case,
>
> Clearly it's the most common case in existing Python code since nothing
> else is allowed. But frankly, even Javascript doesn't follow this
> idiom anymore. Expression-oriented languages have seen a rebirth for
> good reasons (although I admit I'm none-to-fond of many of them, for
> various reasons).
Again, "XYZ language has it" is an ineffective argument.
>> and it allowed us
>> to have extraneous semicolons, braces, or whatever.
>
> Not following this. You mean to *not* have extraneous syntax?
Consider a loop in C vs in python
for (i = 0; i < 50; i++) {
a();
b();
}
for i in range(50):
a()
b()
We avoid the semicolons and braces because the newline and indentation
indicate the same thing. That's what a statement it. Perhaps you can
retain this with your proposal, but only because a statement is still
the default, with expression as a rarely used option.
>> Readability benefits, style consistency benefits.
>
> I strongly disagree. The artificial distinction between statements and
> expressions is the definition of *inconsistent*. Why do we have two
> versions of the "if" conditional? Why do we have "for" loops *and* list
> comprehensions? They express the same ideas, but the limitations in one
> required growing the language another direction. In short, we could
> have had a single, more powerful construct in place of two lesser
> constructs and simultaneously had less syntax to memorize and more
> consistency across the board.
I was referring to consistency of the programs, not the language
itself. No silly arguments about brace position because they *are no
braces*.
There's actually a problem with trying to merge a for-loop and a list
comprehension. A generator expression is the canonical generic form,
but [genexp] would create a list containing a single genexp object.
Likewise, a for-loop would become lazy, so without some extra syntax
to evaluate it (and trigger the side effects!), your programs would
cease to do anything.
So you see, despite significant and obvious similarity between the
features, there's some important differences as well. These are so
obvious from the contexts that you don't even think of them, so
clearly the mental load of having 3 (soon to be 5) different forms of
looping is not all that great.
Mental load is what really counts, not some abstract concept of complexity.
>> Now there are some use cases that suffer here, including the one you
>> just gave: defining a dispatch dict with the functions inline. The
>> best I could do is define the dict first, then stick a decorator on
>> each function to register them. That's still ugly though. A creative
>> solution is needed, but none come to mind.
>
> That's because there is none. And this is my fundamental problem: it's
> not so much that it's hard to do in Python, it's that you *cannot* do it
> in Python. No amount of creativity, time, or experience will help, and
> this is disappointing.
I didn't mean to do it in Python. I meant to modify the language.
> I won't pretend that any example we might toss up in here won't appear
> contrived, but there are definite cases where readability can be
> substantially enhanced with such structures.
>
>> An example where this has happened before is the with-statement, which
>> is spectacularly successful IMO. Now, you may notice it could have
>> been done in a library rather than core syntax if generic anonymous
>> blocks were allowed — so what? The library is still part of the
>> language! It's still something that has to be learned. And the
>> syntax would be substantially uglier using a generic mechanism, rather
>> than the specialized with-statement syntax.
>
> The "so what" is that it could *only* be implemented by the core devs.
> It was not possible for an average (or even above-average) Python
> programmer to write such a library, whereas it *could* have been had the
> language not prohibited it.
As important as it is to extend the language via a library, somewhere
you need to draw the line and start modifying the syntax or other
fundamental builtins. The is a universal tenant, applying even to
lisp (which has very little syntax, rather than extensible syntax).
This is why we have a+b, rather than add(a, b). More syntax when it's worth it.
So it's not that we don't want to allow extensibility - quite the
opposite. It's that we want the common statement to be simpler, and
the extra syntax hasn't been justified for your use cases.
--
Adam Olsen, aka Rhamphoryncus
Josiah Carlson
class dispatcher(dispatch_dictionary):
def x(value):
...
def y(value):
...
def int_23(value):
...
With the proper dispatch_dictionary base class and it's metaclass,
dispatcher would become a dictionary with functions that map from
strings and integers to function handlers.
Never underestimate the power of classes and metaclasses ;) .
- Josiah
Cliff Wells
> On Wed, Sep 10, 2008 at 4:39 PM, Cliff Wells <cl...@develix.com> wrote:
> > On Wed, 2008-09-10 at 15:57 -0600, Adam Olsen wrote:
> >> On Wed, Sep 10, 2008 at 3:18 PM, Cliff Wells <cl...@develix.com> wrote:
> >> > Further, I feel that this limitation forces programmers into using hacks
> >> > and magic or overly spread-out code, which itself leads to readability
> >> > concerns. Having used Python for around a decade, I'm quite aware of
> >> > the fact that you can happily write tons and tons of nice code with
> >> > Python in its current state. However, because of the direction of my
> >> > work (developing a internal DSL in Python) I've suddenly become aware of
> >> > this glass ceiling. I'd bumped into it before back when I was doing a
> >> > lot of GUI development, but blamed it on lambda rather than realizing
> >> > that it wasn't lambda so much as what I am bringing up now.
> >>
> >> Python is not intended for DSLs. Really, don't do it. Python is for
> >> python code.
> >
> > The DSL I work on *is* Python code. And really, this is the first time
> > I've heard anyone assert anything like this. Python is a
> > general-purpose language. It's not VBA ;-)
> >
> > DSL's are an extremely useful concept. To summarily dispatch the whole
> > of them with such an assertion is pretty much bolstering my argument:
> > you've just asserted that Python is inherently limited in scope.
>
> I agree, DSL's are useful. They're just not something python
> currently supports well.
Well, this is where we agree. Where we seem to disagree is whether or
not something needs to be done about it =)
At some level, the whole concept of OO programming *is* DSL support. I
consider "DSL" to mean "mapping of abstract (language) constructs to
real-world (domain-specific) constructs". This type of DSL construction
is quite-well supported by Python (and, in fact, is the way Breve is
constructed under the hood). However, this support is hampered (to
some degree) by having a somewhat inflexible syntax.
> They're the use-case you need to justify
> against the substantial changes you propose.
I guess I don't see it as substantial to people who don't wish to use it
(although it's quite substantial to people who do). Overall, I think
this is why I feel the change doesn't require a huge amount of
justification: you aren't *forced* to use it, but if you need it, it's
huge. It doesn't impose any significant stylistic change on people who
prefer the imperative style, but it opens vast doors for people wishing
to approach problems from a functional path.
> Maybe it's worth it, or
> maybe it's better to add an indent-sensitive string literal that would
> then allow *arbitrary* DSL syntax?
Hm, I'd have to see an example (even contrived) of what you mean here.
>
> >> If you want another language, write your own parser. I
> >> hear lisp is simple to parse, and has no annoying statements to hold
> >> you back!
> >
> > Ah, except Python is the language I like in every way, *except* for this
> > one particular wart. Really, had I not entered new programming domains
> > and tried to take Python with me, I'd probably never have had a
> > complaint.
> >
> > Also, external parsers defeat the entire reasoning behind internal DSL's
> > (taking advantage of an established VM/compiler, requiring users to
> > learn a new syntax in addition to their primary programming language).
>
> I appreciate what you're saying here, and feel much the same way about
> my own pet-features, but this is a really poor argument. Everybody
> has just one wafer-thin feature they'd like to add.
Again, this is only wafer-thin if you don't want to use it (and that is
part of its appeal - it's actually rather non-intrusive to classical
statement-oriented programming).
What I see happening in Python is exactly what you appear to be arguing
against. Little specialized features are added one after the other to
satisfy particular needs, when what is actually needed is one sweeping
change that would make those features redundant.
> >> Seriously though, there is an advantage to basing so much on
> >> statements rather than expressions. We're specialized for one
> >> statement per line, which is the most common case,
> >
> > Clearly it's the most common case in existing Python code since nothing
> > else is allowed. But frankly, even Javascript doesn't follow this
> > idiom anymore. Expression-oriented languages have seen a rebirth for
> > good reasons (although I admit I'm none-to-fond of many of them, for
> > various reasons).
>
> Again, "XYZ language has it" is an ineffective argument.
Not really what I was trying to say. What I was saying is that
expression-oriented languages are rising (practically from the grave C
put them in) because they are inherently useful and typically more
powerful than their imperative counterparts. Where they've tended to
suffer is the place where Python shines: readability. What I'm
claiming is that Python's readability is not due to its imperative
nature, rather due to it's whitespace-oriented syntax and lack of
line-noise, so Python could literally become the best of both worlds
were it to shed its imperative roots.
> >> and it allowed us
> >> to have extraneous semicolons, braces, or whatever.
> >
> > Not following this. You mean to *not* have extraneous syntax?
>
> Consider a loop in C vs in python
>
> for (i = 0; i < 50; i++) {
> a();
> b();
> }
>
> for i in range(50):
> a()
> b()
>
> We avoid the semicolons and braces because the newline and indentation
> indicate the same thing.
That's what I thought you meant. You dropped the "not" in your original
statement.
> That's what a statement it. Perhaps you can
> retain this with your proposal, but only because a statement is still
> the default, with expression as a rarely used option.
Exactly. My point remains that the imperative *style* might still be
preferred (and encouraged) because it *is* inherently simpler to
understand, but that it shouldn't be enforced because it's also
inherently limiting (not surprisingly). I think the community as a
whole has been successful in forwarding "Pythonic" idioms even when the
language allows "unPythonic" coding without resorting to b&d tactics
within the language itself. I think the same external discipline can be
applied to this concept without limiting Python's applicability in other
domains.
> >> Readability benefits, style consistency benefits.
> >
> > I strongly disagree. The artificial distinction between statements and
> > expressions is the definition of *inconsistent*. Why do we have two
> > versions of the "if" conditional? Why do we have "for" loops *and* list
> > comprehensions? They express the same ideas, but the limitations in one
> > required growing the language another direction. In short, we could
> > have had a single, more powerful construct in place of two lesser
> > constructs and simultaneously had less syntax to memorize and more
> > consistency across the board.
>
> I was referring to consistency of the programs, not the language
> itself. No silly arguments about brace position because they *are no
> braces*.
But you are assuming that eliminating statements somehow requires extra
syntax. I think Logix, which was mentioned previously, adequately
demonstrates that this is not the case.
The only "extra" syntax that would need to be used would be parentheses
for logical grouping of expressions. This syntax already exists in
Python and is already used for the same.
> There's actually a problem with trying to merge a for-loop and a list
> comprehension. A generator expression is the canonical generic form,
> but [genexp] would create a list containing a single genexp object.
>
> Likewise, a for-loop would become lazy, so without some extra syntax
> to evaluate it (and trigger the side effects!), your programs would
> cease to do anything.
I don't think so. Think about how Python currently defines a generator:
the presence of the "yield" keyword within a function. I think this
same logic could be applied to a for loop (but I'm willing to be
corrected if you see a problem). A for loop without "yield" is more or
less what it is today (except it might evaluate to []). A for loop with
"yield" is a generator (and by extension, useful as an expression).
> So you see, despite significant and obvious similarity between the
> features, there's some important differences as well. These are so
> obvious from the contexts that you don't even think of them, so
> clearly the mental load of having 3 (soon to be 5) different forms of
> looping is not all that great.
>
> Mental load is what really counts, not some abstract concept of complexity.
This isn't abstract. It's a matter of countable constructs, rules, and
exceptions to rules. Consider:
Rules in statement-oriented Python:
1) distinguishes between statements and expressions
2) expressions return a value, statements do not
3) expressions can be used inside other expressions and statements,
statements cannot
4) there is an if statement
5) there is an if expression (ternary if operator)
6) there are for loop statements
7) there are list comprehensions
8) there are generators ("yield" defines when a function is a generator)
Equivalent rules in expression-oriented Python:
1) if expression returns a value (that can be ignored or used in an
expression)
2) for expression returns an empty list or a list if it's a generator
(has "yield" keyword) that can be ignored or used in an expression.
Which has more "mental load"?
> >> Now there are some use cases that suffer here, including the one you
> >> just gave: defining a dispatch dict with the functions inline. The
> >> best I could do is define the dict first, then stick a decorator on
> >> each function to register them. That's still ugly though. A creative
> >> solution is needed, but none come to mind.
> >
> > That's because there is none. And this is my fundamental problem: it's
> > not so much that it's hard to do in Python, it's that you *cannot* do it
> > in Python. No amount of creativity, time, or experience will help, and
> > this is disappointing.
>
> I didn't mean to do it in Python. I meant to modify the language.
I don't consider modifying the language an acceptable solution for most
programmers. It's a maintenance nightmare.
> >> An example where this has happened before is the with-statement, which
> >> is spectacularly successful IMO. Now, you may notice it could have
> >> been done in a library rather than core syntax if generic anonymous
> >> blocks were allowed — so what? The library is still part of the
> >> language! It's still something that has to be learned. And the
> >> syntax would be substantially uglier using a generic mechanism, rather
> >> than the specialized with-statement syntax.
> >
> > The "so what" is that it could *only* be implemented by the core devs.
> > It was not possible for an average (or even above-average) Python
> > programmer to write such a library, whereas it *could* have been had the
> > language not prohibited it.
>
> As important as it is to extend the language via a library, somewhere
> you need to draw the line and start modifying the syntax or other
> fundamental builtins.
Yes and no. I believe it should be possible to prototype almost any
construct via a library. Whether the language should then embrace the
concept embodied in that prototype to provide better integration,
performance, or simply syntactic sugar, can then be argued much more
fruitfully. If a language prevents you from creating such prototypes
then I think
> The is a universal tenant, applying even to
> lisp (which has very little syntax, rather than extensible syntax).
>
> This is why we have a+b, rather than add(a, b). More syntax when it's worth it.
But the language didn't prevent you from creating the hypothetical add()
function. It merely provided syntactic sugar for making it more
readable.
I feel we're getting slightly side-tracked here =)
> So it's not that we don't want to allow extensibility - quite the
> opposite. It's that we want the common statement to be simpler, and
> the extra syntax hasn't been justified for your use cases.
Once again, I am forwarding *zero* extra syntax. In fact I am
suggesting *less* syntax overall and an accompanying reduction in rules
regarding the remaining syntax. I am suggesting removing (or
deprecating) syntactic additions such as the ternary operator in favor
of extending the power (or more to the point, removing arbitrary
limitations) of the existing core language.
Regards,
Cliff
Josiah Carlson
> On Wed, 2008-09-10 at 17:16 -0600, Adam Olsen wrote:
>> They're the use-case you need to justify
>> against the substantial changes you propose.
>
> I guess I don't see it as substantial to people who don't wish to use it
> (although it's quite substantial to people who do). Overall, I think
> this is why I feel the change doesn't require a huge amount of
> justification: you aren't *forced* to use it, but if you need it, it's
> huge. It doesn't impose any significant stylistic change on people who
> prefer the imperative style, but it opens vast doors for people wishing
> to approach problems from a functional path.
Once language syntax is added to, changed, etc., it's very difficult
to remove those additions, changes, etc., even when the feature is
rarely used, ugly, and generally a bad idea (see back-quotes `x` for
repr(x) ). This may not seem like a big deal to you, because you want
this feature, but for the rest of us who have little (arguably no) use
for the feature, adding semantics to syntax, or adding syntax =
additional mental overhead; never mind the additional sections in the
tutorial where we have to explain why this *particular* special case
of a DSL was special enough to break the rules of explicit is better
than implicit (why should a multi-line if statement implicitly return
something in one place but not in another?)
I know, "it's just one little change". I've made the argument myself.
But that doesn't mean that my idea was a good idea (it wasn't), nor
does it mean that your current idea is (I think everyone in this
thread except you would agree that it's a bad idea).
Before continuing on in defending your proposal here, I suggest you
try comp.lang.python . If you can't get a dozen people to agree that
it's a good idea, or if (like here) the only replies are negative (or
even if the majority of them are), then I would suggest you drop it.
- Josiah
Adam Olsen
Aye, but.. it's hard to explain. There's different levels upon which
you define a language. At the lowest extreme a crude language with no
functions will show patterns in its structure, due to how you use it.
At the highest extreme you change how the underlying parser works, or
maybe even use a graphical language instead. In between is a balance
of how predictable the language is (defining new functions rather than
a new way to *call* functions), with how powerful it is.
You can't simply have both. If the syntax is too open you may as well
stick a new parser in every file, and hope they learn to read it.
>> They're the use-case you need to justify
>> against the substantial changes you propose.
>
> I guess I don't see it as substantial to people who don't wish to use it
> (although it's quite substantial to people who do). Overall, I think
> this is why I feel the change doesn't require a huge amount of
> justification: you aren't *forced* to use it, but if you need it, it's
> huge. It doesn't impose any significant stylistic change on people who
> prefer the imperative style, but it opens vast doors for people wishing
> to approach problems from a functional path.
>
>> Maybe it's worth it, or
>> maybe it's better to add an indent-sensitive string literal that would
>> then allow *arbitrary* DSL syntax?
>
> Hm, I'd have to see an example (even contrived) of what you mean here.
Today, you'd have to use something like this:
foo("""
bar
""")
It's ugly. It uses the equivalent of braces to tell the parser
something that's obvious to us from the indentation. We want it to
look more like a normal block:
foo:
bar
However, this is no longer a function call. You can't put a value
inside parenthesis without moving the parenthesis to the last line,
which is what we don't want. Adding syntax to the language is the
normal solution, but we're looking for something open-ended (that
embeds a string, not python code). We have to cheat somehow. One way
is borrow from decorators:
@foo
blob:
bar
Another is to use $blob$ as a placeholder (only legal when the line is
just an expression, not a statement):
foo($blob$):
bar
>> >> If you want another language, write your own parser. I
>> >> hear lisp is simple to parse, and has no annoying statements to hold
>> >> you back!
>> >
>> > Ah, except Python is the language I like in every way, *except* for this
>> > one particular wart. Really, had I not entered new programming domains
>> > and tried to take Python with me, I'd probably never have had a
>> > complaint.
>> >
>> > Also, external parsers defeat the entire reasoning behind internal DSL's
>> > (taking advantage of an established VM/compiler, requiring users to
>> > learn a new syntax in addition to their primary programming language).
>>
>> I appreciate what you're saying here, and feel much the same way about
>> my own pet-features, but this is a really poor argument. Everybody
>> has just one wafer-thin feature they'd like to add.
>
> Again, this is only wafer-thin if you don't want to use it (and that is
> part of its appeal - it's actually rather non-intrusive to classical
> statement-oriented programming).
>
> What I see happening in Python is exactly what you appear to be arguing
> against. Little specialized features are added one after the other to
> satisfy particular needs, when what is actually needed is one sweeping
> change that would make those features redundant.
Such a sweeping change would only move them into the library, not
remove them from the language. Indeed, because they're forced into an
awkward over-generalized syntax, they become significantly harder to
use.
>> >> Seriously though, there is an advantage to basing so much on
>> >> statements rather than expressions. We're specialized for one
>> >> statement per line, which is the most common case,
>> >
>> > Clearly it's the most common case in existing Python code since nothing
>> > else is allowed. But frankly, even Javascript doesn't follow this
>> > idiom anymore. Expression-oriented languages have seen a rebirth for
>> > good reasons (although I admit I'm none-to-fond of many of them, for
>> > various reasons).
>>
>> Again, "XYZ language has it" is an ineffective argument.
>
> Not really what I was trying to say. What I was saying is that
> expression-oriented languages are rising (practically from the grave C
> put them in) because they are inherently useful and typically more
> powerful than their imperative counterparts. Where they've tended to
> suffer is the place where Python shines: readability. What I'm
> claiming is that Python's readability is not due to its imperative
> nature, rather due to it's whitespace-oriented syntax and lack of
> line-noise, so Python could literally become the best of both worlds
> were it to shed its imperative roots.
Most programmers don't get it. They'd think reducing the axioms in
math from 10 to 5 would make it simpler to learn algebra. Not only
would this have no effect on how they use algebra, they probably can't
even list the axioms anyway!
>> There's actually a problem with trying to merge a for-loop and a list
>> comprehension. A generator expression is the canonical generic form,
>> but [genexp] would create a list containing a single genexp object.
>>
>> Likewise, a for-loop would become lazy, so without some extra syntax
>> to evaluate it (and trigger the side effects!), your programs would
>> cease to do anything.
>
> I don't think so. Think about how Python currently defines a generator:
> the presence of the "yield" keyword within a function. I think this
> same logic could be applied to a for loop (but I'm willing to be
> corrected if you see a problem). A for loop without "yield" is more or
> less what it is today (except it might evaluate to []). A for loop with
> "yield" is a generator (and by extension, useful as an expression).
You're thinking about a generator function, not a generator
expression. A generator expression is the same syntax as a list
comprehension, but it uses () rather rather than [].
Unless you meant to change how generator expressions work.. but that
obviously wouldn't be compatible (nor would any change to a
for-statement).
>> So you see, despite significant and obvious similarity between the
>> features, there's some important differences as well. These are so
>> obvious from the contexts that you don't even think of them, so
>> clearly the mental load of having 3 (soon to be 5) different forms of
>> looping is not all that great.
>>
>> Mental load is what really counts, not some abstract concept of complexity.
>
> This isn't abstract. It's a matter of countable constructs, rules, and
> exceptions to rules. Consider:
>
> Rules in statement-oriented Python:
>
> 1) distinguishes between statements and expressions
> 2) expressions return a value, statements do not
> 3) expressions can be used inside other expressions and statements,
> statements cannot
> 4) there is an if statement
> 5) there is an if expression (ternary if operator)
> 6) there are for loop statements
> 7) there are list comprehensions
> 8) there are generators ("yield" defines when a function is a generator)
>
> Equivalent rules in expression-oriented Python:
>
> 1) if expression returns a value (that can be ignored or used in an
> expression)
> 2) for expression returns an empty list or a list if it's a generator
> (has "yield" keyword) that can be ignored or used in an expression.
>
> Which has more "mental load"?
You're assuming statements and expressions won't continue to be done
in different styles, which just isn't true. Even though a statement
*could* return a value, most of the time it would be used as if it
doesn't, so it remains as a special case to be remembered.
There's also at least two different modes for a for-loop (lazy vs
eager), assuming you drop the list-comp (and 3.0's set/dict
comprehensions).
That's what I meant by "abstract concept of complexity", although
maybe I need a better label for it. You've recategorized thinks to be
the same, yet we're expected to use the same way we always did (other
than occasionally using new functionality). The categories don't
determine complexity, how we use them does!
Your changes actually add a substantial amount of complexity, as well
as being unimplementable (ambiguous or incompatible behaviour). They
never had a chance of being accepted, but I'm trying to explain why
they don't work, so you and other budding language developers might
give better suggestions in the future.
>> >> Now there are some use cases that suffer here, including the one you
>> >> just gave: defining a dispatch dict with the functions inline. The
>> >> best I could do is define the dict first, then stick a decorator on
>> >> each function to register them. That's still ugly though. A creative
>> >> solution is needed, but none come to mind.
>> >
>> > That's because there is none. And this is my fundamental problem: it's
>> > not so much that it's hard to do in Python, it's that you *cannot* do it
>> > in Python. No amount of creativity, time, or experience will help, and
>> > this is disappointing.
>>
>> I didn't mean to do it in Python. I meant to modify the language.
>
> I don't consider modifying the language an acceptable solution for most
> programmers. It's a maintenance nightmare.
If you're not willing to solve it properly then don't come whining to
us. Some problems *need* drastic solutions.
>> >> An example where this has happened before is the with-statement, which
>> >> is spectacularly successful IMO. Now, you may notice it could have
>> >> been done in a library rather than core syntax if generic anonymous
>> >> blocks were allowed — so what? The library is still part of the
>> >> language! It's still something that has to be learned. And the
>> >> syntax would be substantially uglier using a generic mechanism, rather
>> >> than the specialized with-statement syntax.
>> >
>> > The "so what" is that it could *only* be implemented by the core devs.
>> > It was not possible for an average (or even above-average) Python
>> > programmer to write such a library, whereas it *could* have been had the
>> > language not prohibited it.
>>
>> As important as it is to extend the language via a library, somewhere
>> you need to draw the line and start modifying the syntax or other
>> fundamental builtins.
>
> Yes and no. I believe it should be possible to prototype almost any
> construct via a library. Whether the language should then embrace the
> concept embodied in that prototype to provide better integration,
> performance, or simply syntactic sugar, can then be argued much more
> fruitfully. If a language prevents you from creating such prototypes
> then I think
Depends how ugly you're willing to let it get. There's many ways to
do a dispatcher dict, a couple of which have been mentioned already.
--
Adam Olsen, aka Rhamphoryncus
Cliff Wells
> On Wed, Sep 10, 2008 at 6:14 PM, Cliff Wells <cl...@develix.com> wrote:
> > On Wed, 2008-09-10 at 17:16 -0600, Adam Olsen wrote:
> [snip]
> >> They're the use-case you need to justify
> >> against the substantial changes you propose.
> >
> > I guess I don't see it as substantial to people who don't wish to use it
> > (although it's quite substantial to people who do). Overall, I think
> > this is why I feel the change doesn't require a huge amount of
> > justification: you aren't *forced* to use it, but if you need it, it's
> > huge. It doesn't impose any significant stylistic change on people who
> > prefer the imperative style, but it opens vast doors for people wishing
> > to approach problems from a functional path.
>
> Once language syntax is added to, changed, etc., it's very difficult
> to remove those additions, changes, etc., even when the feature is
> rarely used, ugly, and generally a bad idea (see back-quotes `x` for
> repr(x) ). This may not seem like a big deal to you, because you want
> this feature, but for the rest of us who have little (arguably no) use
> for the feature, adding semantics to syntax, or adding syntax =
> additional mental overhead;
Again I assert the opposite: that Python is currently forced to explain
the arbitrary distinction that currently exists and that a significant
amount of extra syntax and language features exist to work around that
distinction. People may not clamor for the change but there's quite a
few newcomers to Python who must have the distinction explained to them.
> never mind the additional sections in the
> tutorial where we have to explain why this *particular* special case
> of a DSL was special enough to break the rules of explicit is better
> than implicit (why should a multi-line if statement implicitly return
> something in one place but not in another?)
I respect your work and knowledge of Python, but that's insufficient to
get me to respect a straw-man argument. I've not suggested that Python
add anything for any particular situation. I'm suggesting a change that
would make it more logically consistent, expressive, and help reduce the
language and syntax clutter it's been accruing at a steady pace. The
fact that it also helps in a rather broad range of problems is ancillary
and my particular use case was presented as an example, not as *the*
problem to be solved.
Also, I don't know where you got the idea that I suggest any expression
should return a value sometimes but not others. I'm suggesting they
*always* return values, but that you don't always need to worry about
them unless you plan to use the value in another expression. Kind of
how functions already work in Python.
> I know, "it's just one little change". I've made the argument myself.
> But that doesn't mean that my idea was a good idea (it wasn't), nor
> does it mean that your current idea is (I think everyone in this
> thread except you would agree that it's a bad idea).
Sure. Because everyone who might have agreed with me has already
migrated to Ruby or started a new language (Logix, Boo, et al). I just
happen to be lazy enough to entertain the thin hope that Python could be
fixed so I don't have to learn a new language or programming
environment ;-)
> Before continuing on in defending your proposal here, I suggest you
> try comp.lang.python . If you can't get a dozen people to agree that
> it's a good idea, or if (like here) the only replies are negative (or
> even if the majority of them are), then I would suggest you drop it.
Actually I avoided c.l.py because I saw this list suggested as the
appropriate place for new or controversial ideas (and the list name
seemed to suggest the same).
The fact that lambda was so bitterly defended against GvR's strong
desire to remove it should be hint enough that a significant number of
Pythonistas are interested in functional and expression-oriented
programming.
In any case, I actually got the response I expected and ultimately I
expect the discussion here was probably far more enlightened than I
would expect on c.l.py.
It seems most Pythoneers work in particular domains where this isn't an
issue, are satisfied with the workarounds, or simply are unable to see
there's a vast world of elegant solutions beyond what imperative
languages can describe. Unfortunately this only confirms my (rather
reluctant) expectation that I'll be forced to move to a more expressive
language to satisfy my programming itches.
In any case, thanks for the feedback.
Regards,
Cliff
George Sakkis
For better or for worse, most people come to Python with imperative rather than functional language background, so the distinction is either not even realized or it usually seems "natural" when realized. Personally, I find some features to be more natural as statements (e.g. it is typically obvious whether to use a "for" loop or a list/gen. comprehension) but others unnecessarily limiting (e.g. having to write "lambda x: x.__setitem__(0,1)" instead of "lambda x: x[0]=1").
George
Josiah Carlson
For people who have gone through the Python tutorial or any one of the
many good 'learning Python' texts (online, paper, free, and paid-for)
the distinction has already been explained. For those who refuse (for
some reason) to go through one of those resources, that's not an issue
with Python, that's an issue with the user.
As a data point; everyone that I've taught Python over the years
(undergrads and interns who only ever used Java before to 30-year
programming veterans of apl/cobol/fortran), not a single one of them
has ever had a problem with the difference between statements and
expressions. It is possible that I've only ever worked with gifted
students and coworkers, though this is unlikely.
>> never mind the additional sections in the
>> tutorial where we have to explain why this *particular* special case
>> of a DSL was special enough to break the rules of explicit is better
>> than implicit (why should a multi-line if statement implicitly return
>> something in one place but not in another?)
>
> I respect your work and knowledge of Python, but that's insufficient to
> get me to respect a straw-man argument. I've not suggested that Python
> add anything for any particular situation. I'm suggesting a change that
> would make it more logically consistent, expressive, and help reduce the
> language and syntax clutter it's been accruing at a steady pace. The
> fact that it also helps in a rather broad range of problems is ancillary
> and my particular use case was presented as an example, not as *the*
> problem to be solved.
What I've found (historically) is that any time I find myself trying
to do something that is not available within the syntax of Python,
it's usually because it's a bad idea. In your case, you want (for
example) the following to be valid Python...
lfcn = lambda x: (
if x:
True
else:
False)
Similarly...
lfcn = lambda x: (
for i in xrange(x):
yield fcn(i, x, ofcn(i))
)
Why? Initially it is for a templating language that you have designed
that makes these kinds of things difficult to do during an automatic
translation from working Python source code into compiled source code.
Presumably something of the form...
html [ body [ a (href='http://python.org') ["Python!"] ] ]
To be converted into something like...
def generate_html(out):
out.write('''<html>
<body>
<a href="http://python.org">Python!</a>
</body>
</html>''')
Trust me, I understand your desire. I have used Cheetah and Spitfire
(http://code.google.com/p/spitfire/), and the speedups gained by
translating templates into Python source are not to be laughed at.
And before I used those, I saw nevow.stan (from which your syntax is
effectively identical to), and wrote my own variant
(http://code.activestate.com/recipes/440563/ - I use multi-calling
semantics rather than __getitem__ stuff). The only reason I didn't
write a compiler (and run into the same issues you have) was because I
didn't care about dynamic content generation, I was using it for
pre-generation of static web pages.
But I digress. At some point, you acknowledge that your templating
language is constrained by your programming language of choice (as is
the case for systems that use the base programming language for
syntax, because of the convenience of parsing), or really, your
templating language is constrained because you refuse to write your
own parser. Don't get me wrong, the convenience of writing a
templating language in Python syntax is amazing...but you've noticed
that your compilation steps are hampered by Python's immediate
execution of code. In fact, your own renderers are a work-around for
something that is /relatively/ natural in Cheetah/Spitfire:
$for $link in $links:
<a href="$link.url">$link.label</a>
Which is generally compiled into the following (with a bit of extra
garbage, but you get the idea)...
for link in links:
out.write('''<a href="%s">%s</a>'''%(link.url, link.label))
Now, I loathe writing Cheetah/Spitfire because I loathe writing html,
which is why I wrote my own templating system, but there are
work-arounds. For example, if you want to stick with Python syntax...
For("$link", "$links", "<a href="$link.url">$link.label</a")
Now you just need to write your For factory and flattener, which
produces the right code (easy), and with a little work, you can even
make for loops composable, add ifs, etc. I know, it's not as elegant
(for you) in this situation as statements becoming expressions, but it
will work today. (note that pie-in-the-sky best-case scenario is 18+
months for Python 3.1, but more likely 15 years for Python 4 ;) )
> Also, I don't know where you got the idea that I suggest any expression
> should return a value sometimes but not others. I'm suggesting they
> *always* return values, but that you don't always need to worry about
> them unless you plan to use the value in another expression. Kind of
> how functions already work in Python.
You are right, I misread an earlier email in this thread.
>> I know, "it's just one little change". I've made the argument myself.
>> But that doesn't mean that my idea was a good idea (it wasn't), nor
>> does it mean that your current idea is (I think everyone in this
>> thread except you would agree that it's a bad idea).
>
> Sure. Because everyone who might have agreed with me has already
> migrated to Ruby or started a new language (Logix, Boo, et al). I just
> happen to be lazy enough to entertain the thin hope that Python could be
> fixed so I don't have to learn a new language or programming
> environment ;-)
That's not necessary. You've already solved your problem with
renderers, and I've pointed out another method for getting some
behavior in-line (If you are careful, you could probably build all of
Python out of constructs similar to the For above). Of course there's
always the option of just using some version of the output of the
compiler/ast (I have done as much to pull out class definitions,
function definitions, etc., in the editor that I have written). You
could even write a tool for doing the conversion ;) .
>> Before continuing on in defending your proposal here, I suggest you
>> try comp.lang.python . If you can't get a dozen people to agree that
>> it's a good idea, or if (like here) the only replies are negative (or
>> even if the majority of them are), then I would suggest you drop it.
>
> Actually I avoided c.l.py because I saw this list suggested as the
> appropriate place for new or controversial ideas (and the list name
> seemed to suggest the same).
>
> The fact that lambda was so bitterly defended against GvR's strong
> desire to remove it should be hint enough that a significant number of
> Pythonistas are interested in functional and expression-oriented
> programming.
Indeed! I have a penchant for the use of list comprehensions,
generator expressions, map, etc. - when they are reasonably
applicable. The question is always: what is reasonably applicable.
> In any case, I actually got the response I expected and ultimately I
> expect the discussion here was probably far more enlightened than I
> would expect on c.l.py.
>
> It seems most Pythoneers work in particular domains where this isn't an
> issue, are satisfied with the workarounds, or simply are unable to see
> there's a vast world of elegant solutions beyond what imperative
> languages can describe. Unfortunately this only confirms my (rather
> reluctant) expectation that I'll be forced to move to a more expressive
> language to satisfy my programming itches.
Pythoneers do see that there are /alternate/ solutions to what is
currently available within Python, hence why people tend to like to
write parsers in Python ;) . One thing you may want to look into is
that you aren't using the full expressive power of Python in your
templating language, and because of this, it is actually quite easy to
write a parser for your subset, adding all of the necessary behavior
and translation. I used DParser in Python for a few years and loved
it, though it seems like the home page for it is no longer available.
On the one hand, yes, you are currently constrained by the limits of
Python's syntax. And yes, you could move to Ruby, Boo, etc. Or you
could try alternate constructs (For, If, Def, etc.), write your own
parser (which isn't hard), or shrug and admit to yourself that the
limitations really aren't all that bad. Inconvenient, sure. The end
of breve in Python? Not necessarily.
- Josiah
Cliff Wells
> On Wed, Sep 10, 2008 at 9:34 PM, Cliff Wells <cl...@develix.com> wrote:
> > On Wed, 2008-09-10 at 18:30 -0700, Josiah Carlson wrote:
> >> On Wed, Sep 10, 2008 at 6:14 PM, Cliff Wells <cl...@develix.com> wrote:
> > Again I assert the opposite: that Python is currently forced to explain
> > the arbitrary distinction that currently exists and that a significant
> > amount of extra syntax and language features exist to work around that
> > distinction. People may not clamor for the change but there's quite a
> > few newcomers to Python who must have the distinction explained to them.
>
> For people who have gone through the Python tutorial or any one of the
> many good 'learning Python' texts (online, paper, free, and paid-for)
> the distinction has already been explained. For those who refuse (for
> some reason) to go through one of those resources, that's not an issue
> with Python, that's an issue with the user.
You won't catch me disagreeing with this, but I was trying (clumsily) to
point out that the distinction violates (albeit subtly) the principal of
least surprise and so requires explanation (even if the explanation is
very simple).
> As a data point; everyone that I've taught Python over the years
> (undergrads and interns who only ever used Java before to 30-year
> programming veterans of apl/cobol/fortran), not a single one of them
> has ever had a problem with the difference between statements and
> expressions. It is possible that I've only ever worked with gifted
> students and coworkers, though this is unlikely.
Without any supporting data, my hunch is that most people who migrate to
Python come from other imperative languages (myself included) and so
don't find the distinction foreign.
Actually, it's much simpler than that. The HTML tags are objects which
simply get serialized into their text representation. I specifically do
not generate Python source. In fact, you can simply do something like
this in the interactive interpreter:
>>> print html [ body [ a (href='http://python.org') ["Python!"] ] ]
>>>
<html><body><a href="http://python.org">Python!</a></body></html>
This is more or less how Nevow Stan does it as well.
I've actually employed several methods for working around this issue in
Breve:
For looping:
- list comprehensions in place of for loops (arguably the natural choice in
Python, but list comps tend toward unreadability quickly).
- tag multiplication e.g.
li(class_='$class')['$value'] * [{'class': 'even', 'value': 1 }, {'class': 'odd', 'value': 2}]
results in <li class="even">1</li><li class="odd">2</li>
For conditionals:
- ternary if-operator for Python >=2.5
- test(val) and [ block ] in place of if-statement (for <2.5 support)
e.g. test (session.logged_in) and div['hello', session.user]
For both:
- renderers - push the logic back into Python proper (as you suggested)
At one point, I even wrote a set of custom tag classes that allowed things like this:
switch ( x ) [
case ( 1 ) [ 'x is 1' ],
case ( 2 ) [ 'x is 2' ],
case ( 3 ) [ 'x is 3' ],
default [ 'x is not in list' ]
]
While these worked superficially, they were ill-fated due to lack of
short-circuiting/lazy evaluation (which rendered them both inefficient
and incorrect).
At the end of the day, while these sorts of methods work, they feel like
what they are: workarounds (although 2.5 helps a bit with the ternary
if-operator). This might be nit-picking on my part, but I don't think
I'm especially unusual in my desire to do the "Right Thing".
Ultimately though, whether/how I solved these issues can be considered
two ways: from a practical point-of-view (what you are espousing) or
from a purist's point-of-view (which I know is not a popular view
amongst Pythonistas). I mentally weighed the pros and cons of having
statements and ended up finding the pros lacking (ultimately coming down
to enforcing an imperative style).
> That's not necessary. You've already solved your problem with
> renderers, and I've pointed out another method for getting some
> behavior in-line (If you are careful, you could probably build all of
> Python out of constructs similar to the For above). Of course there's
> always the option of just using some version of the output of the
> compiler/ast (I have done as much to pull out class definitions,
> function definitions, etc., in the editor that I have written). You
> could even write a tool for doing the conversion ;) .
I've considered this, but frankly find AST manipulation a bit daunting
(and even more of a workaround than the above pure-Python solutions).
I've also considered trying out EasyExtend to accomplish my goals but it
still doesn't feel quite right.
> > The fact that lambda was so bitterly defended against GvR's strong
> > desire to remove it should be hint enough that a significant number of
> > Pythonistas are interested in functional and expression-oriented
> > programming.
>
> Indeed! I have a penchant for the use of list comprehensions,
> generator expressions, map, etc. - when they are reasonably
> applicable. The question is always: what is reasonably applicable.
As you might expect, I subscribe to the "consenting adult" perspective.
I think "reasonably applicable" is a determination best left to the
programmer managing a particular piece of code rather than the
programming language. Readability matters most to those who have to
read it (which is, more often than not, the person who wrote it). What
is readable to one isn't always equally readable to someone else.
For example, list comprehensions used to appear completely inside-out to
me whereas map/filter/reduce was as simple as could be, and yet I've
hear GvR claim on several occasions that the exact opposite is true for
him. Most likely we'd differ just as much on any sufficiently
non-trivial bit of code. But at the end of the day, GvR probably won't
need to worry about how readable any bit of my code is, it only matters
to me. And even if someone else were to need to read my code, it's a
toss-up which way he'd find more readable. Ultimately I don't think
there's a clear win here either way, but I feel that Python needlessly
enforces a single vision to the exclusion of other equally valid
visions.
> > In any case, I actually got the response I expected and ultimately I
> > expect the discussion here was probably far more enlightened than I
> > would expect on c.l.py.
> >
> > It seems most Pythoneers work in particular domains where this isn't an
> > issue, are satisfied with the workarounds, or simply are unable to see
> > there's a vast world of elegant solutions beyond what imperative
> > languages can describe. Unfortunately this only confirms my (rather
> > reluctant) expectation that I'll be forced to move to a more expressive
> > language to satisfy my programming itches.
>
> Pythoneers do see that there are /alternate/ solutions to what is
> currently available within Python, hence why people tend to like to
> write parsers in Python ;) . One thing you may want to look into is
> that you aren't using the full expressive power of Python in your
> templating language, and because of this, it is actually quite easy to
> write a parser for your subset, adding all of the necessary behavior
> and translation. I used DParser in Python for a few years and loved
> it, though it seems like the home page for it is no longer available.
Well, as far as Breve goes, I think there are adequate workarounds.
What has disturbed me was finding I that I actually needed workarounds
(something I haven't needed a lot of in my years of Python programming).
This got me to thinking about the dichotomy of statements and
expressions in Python and I was irrevocably drawn to the conclusion that
this distinction is not as inconsequential as I had once believed.
> On the one hand, yes, you are currently constrained by the limits of
> Python's syntax. And yes, you could move to Ruby, Boo, etc. Or you
> could try alternate constructs (For, If, Def, etc.), write your own
> parser (which isn't hard), or shrug and admit to yourself that the
> limitations really aren't all that bad. Inconvenient, sure. The end
> of breve in Python? Not necessarily.
Admittedly, this is largely a matter of taste. But of course, many
seemingly critical decisions about programming are. Unfortunately, once
I'd had my eyes opened to this limitation, it's now impossible for me
not to notice.
For example, when I see how much more natural Twisted's deferreds appear
in MochiKit than in Python (where they originated), I can't help but
believe that Twisted's failure to reach a larger audience is at least
partially because that approach isn't as easily expressed in Python as
it would be in a more expression-oriented language. In this case, it's
not even that it can't be done (it clearly can), it's that the approach
feels forced rather than natural.
Overall, I'm left with the nagging suspicion that because of my language
choice, I'm missing an important paradigm that would elevate my
programming to the next level.
In any case, I expect I'll stew in my current situation for a while
longer at least. The unfortunate fact remains that most of the
languages that speak to me (e.g. Boo and Io) don't have the broad range
of libraries and frameworks available that Python does and ultimately
this tends to outweigh my esoteric desires.
Regards,
Cliff
Cliff Wells
> On Wed, 10 Sep 2008 15:39:49 -0700
> Cliff Wells <cl...@develix.com> wrote:
> > I strongly disagree. The artificial distinction between statements and
> > expressions is the definition of *inconsistent*. Why do we have two
> > versions of the "if" conditional? Why do we have "for" loops *and* list
> > comprehensions? They express the same ideas, but the limitations in one
> > required growing the language another direction.
>
> initially didn't like list comprehensions. In practice, they're not
> very hard to write, and even easier to read. Doing away with for loops
> because we have them would force you to construct - and then
> immediately throw away - lists in the case where the for loop was
> being executed for side effects.
>
> There have been a number of similar proposals - I've even generated
> some. The general problem is that the results of embedding statements
> with important indentation into expressions is ugly. While I've seen a
> lot of words on this topic, I've seen very few concrete examples - and
> those seem to come as often from opponents to the idea as they do from
> you.
>
> So, how about some concrete examples? Show us translations of the
> various if statements and for loops, so we have something concrete to
> use to judge readability?
Several examples have already been posted in this thread (which has
probably gone on long enough).
Anyway, as was mentioned earlier, Logix very much resembles what I'm
describing:
http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html#3.1
Josiah Carlson
> On Wed, 2008-09-10 at 22:47 -0700, Josiah Carlson wrote:
>> On Wed, Sep 10, 2008 at 9:34 PM, Cliff Wells <cl...@develix.com> wrote:
>> > On Wed, 2008-09-10 at 18:30 -0700, Josiah Carlson wrote:
>> >> On Wed, Sep 10, 2008 at 6:14 PM, Cliff Wells <cl...@develix.com> wrote:
>> As a data point; everyone that I've taught Python over the years
>> (undergrads and interns who only ever used Java before to 30-year
>> programming veterans of apl/cobol/fortran), not a single one of them
>> has ever had a problem with the difference between statements and
>> expressions. It is possible that I've only ever worked with gifted
>> students and coworkers, though this is unlikely.
>
> Without any supporting data, my hunch is that most people who migrate to
> Python come from other imperative languages (myself included) and so
> don't find the distinction foreign.
Your hunch is likely correct. I personally came from Scheme and
C/C++. Many of Python's functional constructs fit the part of me that
got scheme, and everything else fit the part of me that got C/C++.
Not translating to Python source code is a lot less fun ;), and also
makes your templates far less dynamic in real-world scenarios.
[snip]
> Overall, I'm left with the nagging suspicion that because of my language
> choice, I'm missing an important paradigm that would elevate my
> programming to the next level.
Then what the hell are you waiting around here for? If you are
feeling constrained by Python (as a general-purpose language), try
some others! You may want to add Haskell, Caml, or even Lua to your
list of languages to check out. Maybe your Python will get better, or
maybe you'll move on from Python. Who knows?
> In any case, I expect I'll stew in my current situation for a while
> longer at least. The unfortunate fact remains that most of the
> languages that speak to me (e.g. Boo and Io) don't have the broad range
> of libraries and frameworks available that Python does and ultimately
> this tends to outweigh my esoteric desires.
One thing that you need to remember about Python is that it's a
general language. It's syntax and semantics are such that it works in
a fairly large variety of situations. Because it's a general
language, it sometimes doesn't do the things that would be convenient
in a domain specific language. In particular, it doesn't have a lot
of those things that would make functional programming more convenient
(a shorter way of spelling 'lambda', everything is an expression,
etc.), but no one language can be perfect for everyone.
I wish you luck in your adventures with alternate languages.
- Josiah
Cliff Wells
> On Wed, 10 Sep 2008 14:54:44 -0700
> Cliff Wells <cl...@develix.com> wrote:
> > There were several design decisions to be made when designing Breve, the
> > first and foremost being "will Breve templates be full-blown Python
> > programs or will they be limited to expressions?". I chose the latter
> > for a few reasons:
>
> play well with Python code (i.e. - be able to have Breve classes
> inherit from Python classes and vice versa - but I guess without
> statements, you don't have classes, so all of that is moot. Of course,
> not having classes seems like a bad idea as well.
Just to be clear because it appears to me you aren't quite following:
Breve templates *are* Python code, or more exactly, they are a single
Python expression. There is no preprocessor or interpreter that turns
Breve into Python. It's just a bunch of class-magic.
The decision to be a single large expression is largely because this is
a *template* engine and an expression maps most directly to XML because
XML is structured similarly:
html [
head [
title [ 'A Brevé Template' ]
],
body [
h1 [ 'Briefly, Brevé' ], br,
div ( style = 'text-align: center;' ) [
span [ '''
As you can see, Brevé maps very
directly to the final HTML output.
''' ]
]
]
]
This is opposed to allowing the full of Python which would require a lot
of syntax that is extraneous to the markup being generated (and most
importantly would have caused a visual disjoint between the Breve source
template and the final output).
> Further, why do you feel that Breve templates - even if limited to
> expressions - need to be limited to what Python expressions can do?
Again, because Breve *is* Python. It's an internal DSL. Please see
http://martinfowler.com/bliki/DomainSpecificLanguage.html
for a definition of what that means. Another example of an internal DSL
that's more widely used would be SQLAlchemy.
Cliff Wells
> On Thu, 11 Sep 2008 10:40:29 -0700
> Cliff Wells <cl...@develix.com> wrote:
> > Several examples have already been posted in this thread (which has
> > probably gone on long enough).
>
> they don't count.
Since they were more or less exactly what I had in mind, I think they
suffice.
> If you really believe the thread has gone on long enough, then the
> proposal is dead. If you are actually interested in seeing it happen,
> then the real obstacle is that expressions built out of statements
> that use indentation for control flow are *ugly*.
That's only for people not used to seeing them or programming in that
style. Proponents of functional and expression-oriented languages would
strongly disagree. Whether or not they might be ugly *in Python* is a
seemingly separate question, but I don't think it is. In fact, it seems
it would be much more attractive than many traditional FP languages.
> If you have a way
> around it, I'd love to see it. If you don't - then again, the proposal
> is dead.
>
> > Anyway, as was mentioned earlier, Logix very much resembles what I'm
> > describing:
> >
> > http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html#3.1
>
> That link doesn't have any examples, it just repeats many of the words
> that have already been posted.
Yes, the Logix documentation is sorely lacking (I believe the project to
be long-dead), but the intent seemed clear enough to me.
> If you have a link with real live
> examples - in particular, showing how you can use for loops to replace
> lcs and similar things, please provide it.
>From an earlier example I posted:
dispatch = {
'1': lambda x: (
for i in range(x):
if not x % 2:
yield 0
else:
yield 1
),
'2': lambda x: (
for i in range(x):
yield i
)
}
for i in dispatch[val](1):
print i
Overall, if you are familiar with functional programming, it doesn't
take a lot to imagine more examples. If you aren't, then it probably
won't make much sense or seem too appealing in any case.
Cliff Wells
> On Thu, Sep 11, 2008 at 2:05 AM, Cliff Wells <cl...@develix.com> wrote:
> > Overall, I'm left with the nagging suspicion that because of my language
> > choice, I'm missing an important paradigm that would elevate my
> > programming to the next level.
>
> Then what the hell are you waiting around here for? If you are
> feeling constrained by Python (as a general-purpose language), try
> some others! You may want to add Haskell, Caml, or even Lua to your
> list of languages to check out. Maybe your Python will get better, or
> maybe you'll move on from Python. Who knows?
Hey, I said I was lazy! ;-) Anyway, my biggest hurdle to most of the
languages I've considered boils down to:
1) Lua - not expression based
2) Haskell - too ugly
3) Io - no libraries
4) Lisp - I find it borderline unreadable, plus I'd have to talk to
other Lispers
5) Boo - I'd have to learn .NET and deal with the fact that 95% of the
community is on Windows
6) Erlang - too ugly, too fast
7) Ruby - Too ugly, too slow, plus I'd probably have to make a cutesy
website with cartoon animals and robots to show my fandom
Two other options:
1) Logix - I could attempt to revive it. This is actually fairly
appealing if a bit daunting to dive into.
2) Fork Python - I've considered making a proof-of-concept with a Python
fork that does what I want, but it would be disappointing to discard it
when it was inevitably rejected, and I have no intention of becoming a
language maintainer.
I think I'm going to play with Io for a bit in spite its shortcomings as
it is explicitly a multiparadigm language and seems to look the
cleanest.
> > In any case, I expect I'll stew in my current situation for a while
> > longer at least. The unfortunate fact remains that most of the
> > languages that speak to me (e.g. Boo and Io) don't have the broad range
> > of libraries and frameworks available that Python does and ultimately
> > this tends to outweigh my esoteric desires.
>
>
> One thing that you need to remember about Python is that it's a
> general language. It's syntax and semantics are such that it works in
> a fairly large variety of situations. Because it's a general
> language, it sometimes doesn't do the things that would be convenient
> in a domain specific language. In particular, it doesn't have a lot
> of those things that would make functional programming more convenient
> (a shorter way of spelling 'lambda', everything is an expression,
> etc.), but no one language can be perfect for everyone.
But it's so damn close =)
> I wish you luck in your adventures with alternate languages.
Thanks for all your time and thoughts.
Regards,
Cliff
Cliff Wells
> Here's where we disagree. I love functional programming, having
> written 10s of thousands of lines of LISP in the past. The lack of the
> ability to pass around functions in Ruby is why I haven't ever written
> any.
You can:
def a(x,y)
x + y
end
def x(b)
send(b,3,4)
end
puts x(:a) # 7
The semantics are a bit different, but the result is the same (caveat:
I'm no Ruby expert by any stretch of the imagination).
Anyway, not important =)
> And I consider the results of trying to cram python statements
> into expressions as almost universally ugly. I'm willing to consider
> you might have a solution, but you refuse to *show* it to me.
I'm not sure what else you'd like to see. I've provided an example with
a for-loop as generator and if-expressions. I've no intention of
writing a full PEP covering all use-cases at this point.
> > Overall, if you are familiar with functional programming, it doesn't
> > take a lot to imagine more examples. If you aren't, then it probably
> > won't make much sense or seem too appealing in any case.
>
> You're right - it's easy to image in more examples. Really, really
> ugly ones, like:
>
> a = (
> if x < 23:
> 18
> else:
> 24
> ) + (
> if y < 23:
> 29
> else:
> 81
> )
>
> The thought of running into something like the above in production
> code is enough to make me bounce pretty much any proposal that would
> allow it. Compared to that,
>
> a = (18 if x < 23 else 24) + (29 if y < 23 else 81)
a = ( if x < 23: 18 else: 24 ) + ( if y < 23: 29 else: 81 )
Granted, today this is not valid Python, but supporting it doesn't seem
to break existing Python code either. This is precisely how Logix does
it (from the page I linked you to earlier):
x = if 2+2==4: "I thought so" else: "hmmm..."
In any case, your above code seems intentionally formatted to be
unreadable, something that could be achieved under any circumstances.
For example, here's your code reformatted in a fashion similar to your
first example:
a = (
18
if x < 23
else 24
) + (
29
if y < 23
else 81
)
Just as unreadable (although really I don't find it terrible to
comprehend... certainly no worse than a complicated listcomp).
At the end of the day, no programming language is going to be able to
enforce sane coding style (even Python's whitespace can be circumvented
to a large degree).
What we do gain with the idea I've forwarded is a single, more flexible
if-expression in place of an if-statement and an if-operator with
inverted syntax.
Cliff Wells
> On Thu, 11 Sep 2008 12:45:06 -0700
> Cliff Wells <cl...@develix.com> wrote:
> > > The thought of running into something like the above in production
> > > code is enough to make me bounce pretty much any proposal that would
> > > allow it. Compared to that,
> > >
> > > a = (18 if x < 23 else 24) + (29 if y < 23 else 81)
> >
> > a = ( if x < 23: 18 else: 24 ) + ( if y < 23: 29 else: 81 )
>
> indentations in the statement.
Sorry, I'm not going to satisfy "now do it with one hand behind your
back" sort of challenges. This is the syntax I'm proposing, so why
would I provide it any other way?
> Also, you've claimed a number of times
> that you didn't want to change the syntax of python, but this clearly
> requires a syntax change, as current python syntax doesn't allow
> multiple colon-separated statements on a line.
No, I stated I didn't want to *add* any additional syntax (i.e. block
delimiters, keywords or whatnot), nor do I want to break existing code.
I certainly propose changing the grammar (albeit in a demonstrably
backwards-compatible way).
> > Granted, today this is not valid Python, but supporting it doesn't seem
> > to break existing Python code either. This is precisely how Logix does
> > it (from the page I linked you to earlier):
> >
> > x = if 2+2==4: "I thought so" else: "hmmm..."
> >
> > In any case, your above code seems intentionally formatted to be
> > unreadable, something that could be achieved under any circumstances.
> > For example, here's your code reformatted in a fashion similar to your
> > first example:
>
> Yup, and if that was what someone was proposing, it'd be a reason to
> reject it. But nobody is proposing turning expressions into
> statements, which is what you just did.
No, I turned statements into expressions, which is what I've been
proposing since the very first post. I cannot possibly fathom where you
came up with the exact opposite.
You continue to be stuck in a rut of demanding that something be a
statement when this is simply not a requirement. Perfectly legal
(although useless) Python:
1
2
3
Note that this does not somehow magically turn numbers into statements.
It affirms that expressions can be used as statements (something that is
already true in Python). Ask yourself, is a function call a statement
or an expression? It's an expression (a function call *always* returns
a value), and yet it can be used alone:
def foo(x):
return x**2
foo(2) # expression used as a statement
y = foo(3) # expression used as an expression
> > What we do gain with the idea I've forwarded is a single, more flexible
> > if-expression in place of an if-statement and an if-operator with
> > inverted syntax.
>
> But that's not what you've been claiming you wanted. You've been
> claiming you wanted statements to be expressions.
Uh... yeah. Seriously: which part of "this is an example of turning a
particular Python statement into an expression" am I not conveying to
you?
> I claim that doing
> that makes for ugly code, because statements - with newlines and
> whitespace as a critical part of the syntax - are ugly to embed in
> expressions. I've asked for examples and not gotten them. I provide an
> ugly example of statements embedded in an expressions, and you respond
> by providing an alternative trinary operator to rewrite it.
I did not. I'm going to let you try to understand that one line of code
better after you've realized you wouldn't need an operator if the
if-statement were an if-expression.
> I still want to see how you propose to embed *statements* into
> arbitrary expressions, complete with newlines and proper
> indentation. Say...
> a = (
> if x < 24:
> go_make_some_side_effects()
> a_nice_long_function() * another_nice_long_function()
> else:
> go_make_different_side_effects()
> a_twisty_little_function() * a_nice_long_function()
> ) \
> + (
> if y < 23:
> go_make_some_different_side_effects()
> a_twisty_little_function() * another_nice_long_function()
> else:
> go_make_different_side_effects()
> a_nice_long_function() * a_twisty_little_function()
> )
>
> This is still simply bletcherous compared to the obvious legal
> rewrite.
Ignoring your superfluous backslash, the above is a perfectly valid
example. Of course, like all contrived examples, it doesn't necessarily
convey *why* you'd want to do this, but the syntax is correct. Also,
go_make_some_side_effects() is probably ill-advised, and not something
usually done in an FP style which you claim to be familiar with.
Imperative programming is all about side-effects whereas functional
programming is all about avoiding them.
> But it's about the cleanest way I've seen to deal with the
> issue. Using operator.add might be a bit cleaner, but not enough so to
> justify using operator.add.
>
> And we haven't gotten *really* warped yet, with statements appearing
> as controlling expressions inside of other statements. While making
> this "simple" changes allows you to make a few use cases simpler, it
> also makes some truly ugly things seem reasonable. Frankly, I'd rather
> let the experts deal with those use cases individually with
> well-thought out extensions than open this can of worms - unless you
> can show how to do those kinds of embeddings in ways that *aren't*
> ugly.
I point you to the entire history of FP programming and we can leave it
at that. If you've programmed in Lisp as you claim then I cannot see
any reason to rehash what you, by definition, must already know, but
keep pretending not to.
Unless you're willing to admit you simply don't know FP or that you are
being intentionally obtuse for some unfathomable reason, then I'm not
interested in pursuing this discussion with you any further. If it's a
matter of me not being clear, then I apologize, but frankly I've been as
clear as I am able so further discussion would still certainly seem to
be fruitless.
Adam Olsen
> On Thu, 2008-09-11 at 16:21 -0400, Mike Meyer wrote:
>> Also, you've claimed a number of times
>> that you didn't want to change the syntax of python, but this clearly
>> requires a syntax change, as current python syntax doesn't allow
>> multiple colon-separated statements on a line.
>
> No, I stated I didn't want to *add* any additional syntax (i.e. block
> delimiters, keywords or whatnot), nor do I want to break existing code.
> I certainly propose changing the grammar (albeit in a demonstrably
> backwards-compatible way).
We keep disagreeing on this point. You're not adding any new tokens,
but you are adding a ton of new syntax!
Not to mention that doing it generically doesn't work (a for-loop
needs to evaluate eagerly, but mustn't retain all the values it
contained), and your primary example (dispatch-dict) only needs lambda
to allow statements, rather than the broad changes you suggest.
--
Adam Olsen, aka Rhamphoryncus
Adam Olsen
> Also,
> go_make_some_side_effects() is probably ill-advised, and not something
> usually done in an FP style which you claim to be familiar with.
> Imperative programming is all about side-effects whereas functional
> programming is all about avoiding them.
This isn't true when applied to python. We're about halfway in
between, regularly avoiding side effects (immutable int/str, sorted(),
iteration is generic and only covers reading), while happily doing
side-effects when appropriate.
Moreover, although iteration is built on mutation (the iterator
object's state changes as you go through it), good style is to contain
that in a single function. The end result is often directly
translatable into a side-effect-free language, only our version is
easier to read; all they offer is a guarantee of no side effects.
--
Adam Olsen, aka Rhamphoryncus
Bruce Leban
Terry Reedy
>> If you have a link with real live
>> examples - in particular, showing how you can use for loops to replace
>> lcs and similar things, please provide it.
>
>>From an earlier example I posted:
>
> dispatch = {
> '1': lambda x: (
> for i in range(x):
> if not x % 2:
> yield 0
> else:
> yield 1
> ),
>
> '2': lambda x: (
> for i in range(x):
> yield i
> )
> }
>
> for i in dispatch[val](1):
> print i
To me, this is the most (perhaps only ;-) readable such example. But it
is unnecessary. Even without metaclasses, as someone else suggested
(and thank you for the idea), the following works today, and is even
more readable to me.
class disclass(): # 3.0
def f1(x):
for i in range(x):
if not x % 2:
yield 0
else:
yield 1
def f2(x):
for i in range(x):
yield i
dispatch = disclass.__dict__
val = '1'
for i in dispatch['f'+val](2):
print(i)
for i in dispatch['f'+val](3):
print(i)
val = '2'
for i in dispatch['f'+val](4):
print(i)
>>>
0
0
1
1
1
0
1
2
3
In 3.0, a class decorator could, I believe, both replace the class with
its dict, and strip the leading 'f' from the 'fn' keys. So there is
another 'need' for generalized lambda that isn't.
Terry Jan Reedy
Cliff Wells
> I agree that making every statement also an expression has merits, as
> does FP. However, you're handwaving about how this fits into python.
Sure. I intentionally handwave for two reasons:
1) it's a bit of a waste of time to detail something like this when it's
universally rejected
2) I didn't want to bicker over details. I wanted to forward the
concept.
> The scope of an if is defined syntactically by indentation. Your
> proposal changes that in some way that I can't quite tell. Is it that
> you can put a single expression between the if and the else? Or
> something else? When would line breaks be optional where they are now
> required?
I'd say anytime it's ambiguous (i.e. much like you would for
parenthesizing any other group of expressions).
Logix manages some of the ambiguity by defining ';' as an operator that
simply evaluates to the rightmost value. I think this makes a lot of
sense.
if x: y else: z # seems obvious
if x: y; y+1 else: z; z+1 # evaluates to y+1 or z+1 depending on x
and is equivalent to
if x:
y
y+1
else:
z
z+1
A the primary question that arises is what to make of assignment. GvR
is notoriously against allowing constructs such as:
if ( x=1 ): y
and I think his reasoning is sound (this is usually a programming
error). I tend to think it should allowed (for consistency), but it
could be defined as a syntax error in this context. At any rate, I'll
resort to the same argument as is used against people who demand static
type declarations: unit testing =). Frankly, either is acceptable to
me.
One thing that would have to change about assignment is it would
definitely *have* to be an expression (and evaluate to the assigned
value). Whether it's allowed in any particular context is open.
> Simply saying "make everything an expression" is insufficient. Saying
> "it's just lifting a restriction" is also insufficient. In many
> languages there are restrictions on using the + operators with
> non-numbers. Why can't we just remove that restriction? Well, we have
> to define the precise semantics in order to do that, e.g., what does
> []+{} mean?
>
>
> You said that the value of a for loop would have the value [] if it
> didn't include yield, i.e.,
> for x in y: foo(x)
>
> evaluates to [] while you said
> if x: foo(x) else: foo(0)
> evaluates to foo(something). That seems inconsistent.
> Is that inconsistency a good thing? For that matter, why return a list
> rather than () or an empty generator? [[As an aside: why can't I write
> {foo(x) for x in y} which is the equivalent of doing d[x] = foo(x) for
> all x. I've just written the equivalent loop several times recently.]]
Both evaluate to *something*, just that the non-yielding for-loop
evaluates to an empty list (probably a good indication it's not being
used in an enclosing expression). It could probably evaluate to None,
if that seemed more reasonable. I tend to prefer an empty list as it
allows for looping without testing for an iterable. Incidentally, how
would an empty generator differ from []? I'll concede it might make
more logical sense, but would it differ in application?
> To answer these questions, the right thing to do is to write a list of
> exactly what new syntax and semantics you are proposing. I'm not
> saying that I (or anyone else) will agree with what you propose but I
> certainly can't agree with the vague proposal you've made so far.
I'd probably defer heavily to an existing implementation that does
exactly what I'm referring to (except I'm not espousing macros or other
metaprogramming facilities Logix provides):
http://www.livelogix.net/logix/index.html
Whether or not this is the ideal situation is arguable, but those
arguments would need to be much further down the road. Right now
there's not even agreement on whether the overarching concept is even
appropriate for Python.
Mathias Panzenböck
functions:
class A(object):
def some_method(self):
pass
def function_that_wants_a_A(a):
...
function_that_wants_a_A(A():
def some_method(self):
# I really like this feature of java
# granted, it makes a lot more sense in the
# context of a language like java.
# there it's very handy
print "this is my derived anonymous class"
)
def function_that_wants_a_callback(callback):
...
function_that_wants_a_callback(def(a,b,c):
# do a lot of stuff that does not fit into a lambda
# this reminds a bit of rubies closures ;)
)
I don't think anything else concerning statement as expression is in any way
necessary or handy.
Cliff Wells
> On Thu, Sep 11, 2008 at 3:06 PM, Cliff Wells <cl...@develix.com> wrote:
> > Also,
> > go_make_some_side_effects() is probably ill-advised, and not something
> > usually done in an FP style which you claim to be familiar with.
> > Imperative programming is all about side-effects whereas functional
> > programming is all about avoiding them.
>
> This isn't true when applied to python. We're about halfway in
> between, regularly avoiding side effects (immutable int/str, sorted(),
> iteration is generic and only covers reading), while happily doing
> side-effects when appropriate.
Well, I'd argue that *any* statement is causing a side-effect as it
affects the flow of control or values of objects outside its own scope.
Even the following represents one side-effect (defining a class in the
outer scope):
class Foo(object):
pass
Expressions on the other hand, *evaluate* into the outer scope. They
must be explicitly assigned to have any long-term effect once they've
completed.
One way to look at it is that statements "push" values into the
enclosing scope whereas expressions "offer" values to it. I'd call the
"pushing" a side-effect. Note that I am not claiming side-effects to be
universally bad things (they often are, but they are just as often
unavoidable to do real work, i.e. printing to the terminal).
> Moreover, although iteration is built on mutation (the iterator
> object's state changes as you go through it), good style is to contain
> that in a single function. The end result is often directly
> translatable into a side-effect-free language, only our version is
> easier to read; all they offer is a guarantee of no side effects.
Not 100% I'm sure I follow this, but I think I might agree ;-)
Cliff
Cliff Wells
> The only thing I'd like would be anonymous classes (like in java) and anonymous
> functions:
> I don't think anything else concerning statement as expression is in any way
> necessary or handy.
At one time I also perceived a shortcoming in lambda. You could
continue to see it this way, or you could see it (like I now do) that
it's not lambda that's limited (it effectively allows an arbitrary
number of expressions), but rather the fact that a statement cannot be
be used as an expression.
For instance, the following is currently legal in Python:
lambda x, y: (
x + y,
x**2,
y**2
)
But this is not:
lambda x, y: (
if x > y: x
else: y
)
Whether or not it's illegal is due to lambda being limited or because
statements are castrated expressions is all a matter of your point of
view. Extending lambda to allow statements would certainly fix
*lambda*, but making statements into expressions fixes Python (IMHO, of
course).
Regards,
Cliff
Guido van Rossum
I think the only way to move forward in this discussion is if you
could write a complete parser for the variant of Python that you'd
like to see.
I expect that you'll say "but I don't know enough about parser
technology to do so" -- but then my response is "then you don't know
to understand the difficulty of the task, so shut up." If you want to
continue this discussion you'll have to prove that it is possible.
That in turn will force you to think about many of the details that
you're currently brushing away as irrelevant -- and you'll find that
they are actually highly relevant.
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
Adam Olsen
> On Thu, 2008-09-11 at 15:53 -0600, Adam Olsen wrote:
>> On Thu, Sep 11, 2008 at 3:06 PM, Cliff Wells <cl...@develix.com> wrote:
>> > Also,
>> > go_make_some_side_effects() is probably ill-advised, and not something
>> > usually done in an FP style which you claim to be familiar with.
>> > Imperative programming is all about side-effects whereas functional
>> > programming is all about avoiding them.
>>
>> This isn't true when applied to python. We're about halfway in
>> between, regularly avoiding side effects (immutable int/str, sorted(),
>> iteration is generic and only covers reading), while happily doing
>> side-effects when appropriate.
>
> Well, I'd argue that *any* statement is causing a side-effect as it
> affects the flow of control or values of objects outside its own scope.
In a strict technical sense, sure. Perhaps it's better to say that
python encourages *containing* side-effects.
--
Adam Olsen, aka Rhamphoryncus
Cliff Wells
> Hey Cliff,
>
> I think the only way to move forward in this discussion is if you
> could write a complete parser for the variant of Python that you'd
> like to see.
As I've pointed out, this has already been done (although the project
appears dead):
http://www.livelogix.net/logix/index.html
http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html
Logix is actually much more than a Python interpreter, but that's
tangential to this discussion. Logix also conveniently runs on the
Python VM. It also removes the distinction between statements and
expressions and is nearly code-compatible with Python (it has two
significant shortcomings and two minor ones, none of which have to do
with the topic at hand).
http://www.livelogix.net/logix/tutorial/3-Introduction-For-Python-Folks.html
> I expect that you'll say "but I don't know enough about parser
> technology to do so" -- but then my response is "then you don't know
> to understand the difficulty of the task, so shut up."
> If you want to continue this discussion you'll have to
> prove that it is possible.
Or I can continue to say what I like and you can choose to ignore it. I
obviously believe it's a worthwhile discussion and certainly enjoy the
debate, but anyone who doesn't is best served by not joining it. I
don't recall bringing this to python-dev as a serious proposal nor
submitting it as a PEP, so I think an informal exchange of ideas without
spending six months on a functional prototype is perfectly acceptable.
That aside, it's already been proven that the syntax and concept is
plausible in Logix. Whether or not it could be implemented in CPython
is another matter, and whether or not the result would be accepted is
yet a another matter.
> That in turn will force you to think about many of the details that
> you're currently brushing away as irrelevant -- and you'll find that
> they are actually highly relevant.
I'm not brushing them away as irrelevant, I'm setting them aside until
it's decided that it's even worth pursuing. I don't know about you, but
I don't bother arguing with my wife about the color of a new cars until
we're certain we need to buy one.
I certainly agree that if I decided to move this beyond idle discussion
that a prototype would be a critical aspect and I fully expect there
would be many issues to be overcome. I don't believe such a change
would be simple, but neither do I believe it's insurmountable.
Regards,
Cliff
Guido van Rossum
Well, if you are asking for my personal opinion about the viability of
this idea, I think it's not viable, but would like to be proven wrong
(or right!) just so that this topic can be put to bed for good. I'm
not swayed by the existence of Logix; it appears to be using a
completely different way of interpreting whitespace, which seems
incompatible with current Python, but they don't seem to give the
exact rules, only some hints. Have you used it?
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
Cliff Wells
> On Thu, Sep 11, 2008 at 9:08 PM, Cliff Wells <cl...@develix.com> wrote:
> > I certainly agree that if I decided to move this beyond idle discussion
> > that a prototype would be a critical aspect and I fully expect there
> > would be many issues to be overcome. I don't believe such a change
> > would be simple, but neither do I believe it's insurmountable.
>
> Well, if you are asking for my personal opinion about the viability of
> this idea, I think it's not viable, but would like to be proven wrong
> (or right!) just so that this topic can be put to bed for good.
Well, that was actually my goal in bringing it up here: I've got
something nagging me and the only way I'm going to put it to rest is to
hash it out in a roomful of devil's advocates =)
> I'm not swayed by the existence of Logix; it appears to be using a
> completely different way of interpreting whitespace, which seems
> incompatible with current Python, but they don't seem to give the
> exact rules, only some hints. Have you used it?
I've imported Logix into a Pylons session and used some of it's features
without issue. It's quite probable that this may have only been
successful because Pylons itself was precompiled with CPython and the
only code affected was the particular module I was compiling at the
time.
I'm curious which aspect of Logix' whitespace handling you suspect to be
incompatible (aside from the line-continuation stuff).
In any case, as fascinating as Logix is, I find it mostly interesting
(to this debate) as an example of what such a syntax could appear like.
Nevertheless, I think I have a compromise: in lieu of writing an
interpreter, I'll investigate and properly document Logix (something
I've considered in the past anyway). This way we can at least have an
implementation to argue over without me needing to quit my day job to
make a point ;-)
Regards,
Cliff
Cliff Wells
> Nevertheless, I think I have a compromise: in lieu of writing an
> interpreter, I'll investigate and properly document Logix (something
> I've considered in the past anyway). This way we can at least have an
> implementation to argue over without me needing to quit my day job to
> make a point ;-)
Bah, it appears it doesn't work under 2.5 (I haven't tried it since
2.4). I guess my work is cut out for me :P
Scott Dial
> Sure. I intentionally handwave for two reasons:
> 1) it's a bit of a waste of time to detail something like this when it's
> universally rejected
And yet this thread, persists.. *sigh*
-Scott
--
Scott Dial
sc...@scottdial.com
sco...@cs.indiana.edu
Scott Dial
>> That in turn will force you to think about many of the details that
>> you're currently brushing away as irrelevant -- and you'll find that
>> they are actually highly relevant.
>
> I'm not brushing them away as irrelevant, I'm setting them aside until
> it's decided that it's even worth pursuing. I don't know about you, but
> I don't bother arguing with my wife about the color of a new cars until
> we're certain we need to buy one.
I'm sorry, but the points people are bringing up are not bikeshedding.
They are very real points that are more analogous to whether the car has
wheels or flies on pixie dust. I'm sorry, but you can't cast aside the
subtleties of turning statements into expressions as bikeshedding
arguments. Many of the people who gave you very genuine feedback pointed
out particularly difficult to rectify problems (like how exactly do you
notate a suite if you eliminate the required indent tokens?) and your
response to them is that they are missing the point.. You want to change
the language, the syntax of that change is *exactly* the point.
-Scott
--
Scott Dial
sc...@scottdial.com
sco...@cs.indiana.edu
Cliff Wells
> Cliff Wells wrote:
> >> That in turn will force you to think about many of the details that
> >> you're currently brushing away as irrelevant -- and you'll find that
> >> they are actually highly relevant.
> >
> > I'm not brushing them away as irrelevant, I'm setting them aside until
> > it's decided that it's even worth pursuing. I don't know about you, but
> > I don't bother arguing with my wife about the color of a new cars until
> > we're certain we need to buy one.
>
> I'm sorry, but the points people are bringing up are not bikeshedding.
> They are very real points that are more analogous to whether the car has
> wheels or flies on pixie dust.
Fair enough. I simply think there's a larger philosophical issue: is
the potential for deeply nested code a greater concern than an
ever-expanding suite of special-case constructs? That needs to be
agreed on before deciding on a specific way of addressing it.
Even if we agreed on the larger issue I've forwarded, what I'm
suggesting may not be the only way of addressing it.
> I'm sorry, but you can't cast aside the
> subtleties of turning statements into expressions as bikeshedding
> arguments. Many of the people who gave you very genuine feedback pointed
> out particularly difficult to rectify problems (like how exactly do you
> notate a suite if you eliminate the required indent tokens?)
To be clear, I don't suggest eliminating the required indentation for
multi-line blocks (although it seems you could intentionally dodge it
through judicious use of the ';' operator and parentheses, much like you
can with expressions and operators today).
> and your
> response to them is that they are missing the point.. You want to change
> the language, the syntax of that change is *exactly* the point.
Well, I'll take the blame for not spending more time separating out the
fact that I'm presenting both a problem and a potential solution which
has led to a conflation of the two to some degree. I think the solution
I've presented helps demonstrate that there *is* a problem and so I find
it difficult to present the problem in the solution's absence.
As I mentioned in my first post, the current situation has arisen
because of a philosophy ("flat is better than nested"). This seems a
valid position, but I'm arguing that it has proven over time to have as
many pitfalls (in terms of complexity) as the problem it seeks to avoid.
Hopefully that makes it more clear why I'm trying to get people to see
(and hopefully agree) that there's a problem before bickering over the
details of its solution.
Regards,
Cliff
Leonardo Santagada
Fair enough. I simply think there's a larger philosophical issue: is
the potential for deeply nested code a greater concern than an
ever-expanding suite of special-case constructs? That needs to be
agreed on before deciding on a specific way of addressing it.
Leonardo Santagada
Fair enough. I simply think there's a larger philosophical issue: is
the potential for deeply nested code a greater concern than an
ever-expanding suite of special-case constructs? That needs to be
agreed on before deciding on a specific way of addressing it.
Cliff Wells
> On Thu, 11 Sep 2008 18:01:55 -0700
> Cliff Wells <cl...@develix.com> wrote:
>
> > On Thu, 2008-09-11 at 15:14 -0700, Bruce Leban wrote:
> > > I agree that making every statement also an expression has merits, as
> > > does FP. However, you're handwaving about how this fits into python.
> >
> > Sure. I intentionally handwave for two reasons:
> >
> > 1) it's a bit of a waste of time to detail something like this when it's
> > universally rejected
> > 2) I didn't want to bicker over details. I wanted to forward the
> > concept.
>
> community. Clearly, that's not the case.
I'll concede I more-or-less stopped reading c.l.py at least 3 or 4 years
ago ;-)
> If you look through the
> proposed changes to python, you'll find this kind of thing (changes
> that introduce a single powerful feature that let you manipulate
> statements in expressions) surfacing at regular intervals over at
> least the past 15 years. The idea is almost never rejected
> outright. In fact, if you look, that's what's happening here - people
> aren't rejecting your position outright, but trying to get enough
> information to figure out how it's going to affect the entire
> language.
Yes, I accept the responsibility for conflating what I see as a problem
with a possible solution to that problem (the two were inextricably
linked in my mind).
> That's another place where the Python community differs from other
> language communities. We don't believe in features for the sake of
> features (whether you get them by adding or removing things); we
> believe the features in the language should interact well with each
> other. So while some feature may scratch a particular itch - say to do
> FP programming in Python - the fact that it turns ugly when used in
> non-FP code, or requires ignoring fundamental language layout rules,
> are not things that tie one hand behind your back, but crucial
> problems that must be solved if the feature is to be added. So even
> though the idea behind a feature may be generally acceptable - and
> that seems to be the case here - it won't be adopted until those
> details get worked out.
Agreed, and I had intended to move to that stage once I felt there was a
consensus that there is actually something that needs solving. What I
failed to make clear in my position is that I'm not trying to add a
feature for a it's own sake, rather that the current process for
limiting the growth of the Python language is doomed to failure.
I'm going to try to rephrase the fundamental problem I see here and
hopefully I won't fail so dismally this time:
1) I assert that small is better than large when it comes to limiting
complexity.
2) The primary method of limiting Python's core feature growth is for
the BDFL to dig in his heels and reject ideas until they are adequately
shown to be broadly needed and implementable.
3) This method has failed in the past*. In fact, I assert that this
method is guaranteed to fail unless all ideas that add syntactical
structures are rejected outright, regardless of their utility. It
cannot limit the growth of Python's core, it can only limit the rate at
which it grows.
4) As Python moves into new domains, as new programming languages come
into vogue, and as the art of programming itself advances (albeit
glacially), users will perceive needs in Python and will clamor for
extensions. Some of these will eventually be accepted.
5) Given the added desire to maintain backwards-compatibility, old
features will not be shed as fast as new ones are added (unless that
becomes part of the process, which doesn't seem practicable to me).
6) I believe that a large class of these features could be rendered moot
or redundant if the language embraced a more sweeping and fundamental
change. While this won't absolutely prevent the language's growth, it
provides a built-in deterrent.
* See the ternary if-operator for an example - steadily rejected by GvR
for many years and then finally accepted, probably out of exhaustion.
There's an old quote by Larry Wall: "Perl is ugly because people wanted
it that way". Perl took the same approach to limiting features that
Python does, with the notable difference that Larry didn't reject ideas
as consistently as Guido (or apparently at all). Nevertheless, we're
now simply discussing the *rate* at which the language becomes large and
inconsistent rather than whether it will or not.
If we agree that the issues I outline above are valid, then I think we
can start bickering over possible solutions and how those solutions
would affect Python on the whole.
I apologize for my previous response to you. It was clearly my own
failing to properly explain my position that led to our exchange.
Regards,
Josiah Carlson
The growth limiting mechanism is doomed to failure, or the language is
doomed to failure?
> I'm going to try to rephrase the fundamental problem I see here and
> hopefully I won't fail so dismally this time:
>
> 1) I assert that small is better than large when it comes to limiting
> complexity.
It would seem that you are saying that limiting Python's complexity is
a bad thing. Please correct me if my understanding of your statement
is incorrect.
I'm personally of the opinion that limiting the complexity of Python
as a language is a good thing. And so far, I think Guido has done a
fairly very good job at keeping Python moving in a positive direction.
> 2) The primary method of limiting Python's core feature growth is for
> the BDFL to dig in his heels and reject ideas until they are adequately
> shown to be broadly needed and implementable.
>
> 3) This method has failed in the past*. In fact, I assert that this
> method is guaranteed to fail unless all ideas that add syntactical
> structures are rejected outright, regardless of their utility. It
> cannot limit the growth of Python's core, it can only limit the rate at
> which it grows.
If the rate of language feature growth is slowed by Guido, then
obviously the growth of Python's core language is limited. I would
argue that this is a good thing, as the more language syntax and
features that are present, the steeper the learning curve for new
users of the language.
Python 2.5.2 is a wholly different beast from the Python 1.5.2 that I
started with, but I've had nearly 10 years to update my understanding
of the language. For new users? Metaclasses, list comprehensions,
generator expressions, generators, ... I get asked about them all.
And the language keeps growing!
> 4) As Python moves into new domains, as new programming languages come
> into vogue, and as the art of programming itself advances (albeit
> glacially), users will perceive needs in Python and will clamor for
> extensions. Some of these will eventually be accepted.
But not all. Why? Because not all proposed extensions to Python are
worth discussing outside of a water-cooler "wouldn't it be neat if?"
discussion. See the proposed extension to list comprehensions where
"order by" was proposed from sql (a year or two ago at this point).
That shouldn't have even made it to python-dev. Yet it did. Why?
Because everyone has their "I really want X", and too few people are
willing to acknowledge that they can be wrong.
> 5) Given the added desire to maintain backwards-compatibility, old
> features will not be shed as fast as new ones are added (unless that
> becomes part of the process, which doesn't seem practicable to me).
Python 3.0 is one of the ways that features are shed. While Guido has
stated that he doesn't believe that there will be a Python 4k (aka
Python 4.0) that hits the reset button again, I think that there needs
to be one, so that Python *can* get rid of bad features that we were
wrong about. There's also the PEP process, which as long as there
isn't sweeping "break everything" changes, it would seem that we can
remove features as time goes on.
> 6) I believe that a large class of these features could be rendered moot
> or redundant if the language embraced a more sweeping and fundamental
> change. While this won't absolutely prevent the language's growth, it
> provides a built-in deterrent.
...To a functional programming paradigm. That's what you meant to say, right?
> There's an old quote by Larry Wall: "Perl is ugly because people wanted
> it that way". Perl took the same approach to limiting features that
> Python does, with the notable difference that Larry didn't reject ideas
> as consistently as Guido (or apparently at all). Nevertheless, we're
> now simply discussing the *rate* at which the language becomes large and
> inconsistent rather than whether it will or not.
>
> If we agree that the issues I outline above are valid, then I think we
> can start bickering over possible solutions and how those solutions
> would affect Python on the whole.
I don't agree.
Adam Olsen
> 1) I assert that small is better than large when it comes to limiting
> complexity.
We seem to have a fundamental disagreement here. I insist your
changes would add substantial complexity with little benefit,
primarily adding more ways to accomplish things that can easily be
already accomplished.
We are at an impasse.
--
Adam Olsen, aka Rhamphoryncus
Cliff Wells
> On Fri, Sep 12, 2008 at 8:55 AM, Cliff Wells <cl...@develix.com> wrote:
> > Agreed, and I had intended to move to that stage once I felt there was a
> > consensus that there is actually something that needs solving. What I
> > failed to make clear in my position is that I'm not trying to add a
> > feature for a it's own sake, rather that the current process for
> > limiting the growth of the Python language is doomed to failure.
>
> The growth limiting mechanism is doomed to failure, or the language is
> doomed to failure?
I'm asserting that the limiting mechanism is doomed to failure.
> > I'm going to try to rephrase the fundamental problem I see here and
> > hopefully I won't fail so dismally this time:
> >
> > 1) I assert that small is better than large when it comes to limiting
> > complexity.
>
> It would seem that you are saying that limiting Python's complexity is
> a bad thing. Please correct me if my understanding of your statement
> is incorrect.
No, I'm saying limiting complexity is good and that smaller is better.
> I'm personally of the opinion that limiting the complexity of Python
> as a language is a good thing. And so far, I think Guido has done a
> fairly very good job at keeping Python moving in a positive direction.
Yes, but I also feel that the language has already grown to a point
where only a shrinking subset of Python users fully know and understand
the full feature set. For people who have been using Python for many
years, they've possibly had time to digest the new features. I don't
believe new users fare as well.
> > 2) The primary method of limiting Python's core feature growth is for
> > the BDFL to dig in his heels and reject ideas until they are adequately
> > shown to be broadly needed and implementable.
> >
> > 3) This method has failed in the past*. In fact, I assert that this
> > method is guaranteed to fail unless all ideas that add syntactical
> > structures are rejected outright, regardless of their utility. It
> > cannot limit the growth of Python's core, it can only limit the rate at
> > which it grows.
>
> If the rate of language feature growth is slowed by Guido, then
> obviously the growth of Python's core language is limited.
A slow boat still reaches its destination. My position is that by
obviating the need for a large class of syntax extensions, the onus is
moved from the Python core team onto the user desiring a particular
extension and by doing so, things like community pressure to add X can
be reduced and by extension, the likelihood of the process folding under
that pressure becomes much less likely.
> I would argue that this is a good thing, as the more language syntax and
> features that are present, the steeper the learning curve for new
> users of the language.
Absolutely. But this isn't what Python has been doing (as you
acknowledge below). We seem to be agreeing on this point, so I believe
our disagreement must center around whether the current process
continues to be sufficient.
> Python 2.5.2 is a wholly different beast from the Python 1.5.2 that I
> started with, but I've had nearly 10 years to update my understanding
> of the language. For new users? Metaclasses, list comprehensions,
> generator expressions, generators, ... I get asked about them all.
> And the language keeps growing!
I switched from C to Python right as 1.5.1 was being phased out. To
this day I continue to use a subset of Python that is probably much
closer to 1.5.x than 2.5. This may be attributed to a personal failing
(my self-professed laziness), but the fact remains that I prefer a
small, versatile set of constructs to a large set of specialized ones.
C, for all of its shortcomings, had one absolutely compelling feature:
you could keep the entire language in your head and you could do it in
an amazingly short time. Python 1.5 was similar in scope. 2.5 is
arguably at or beyond that limit for most people.
> > 4) As Python moves into new domains, as new programming languages come
> > into vogue, and as the art of programming itself advances (albeit
> > glacially), users will perceive needs in Python and will clamor for
> > extensions. Some of these will eventually be accepted.
>
> But not all. Why? Because not all proposed extensions to Python are
> worth discussing outside of a water-cooler "wouldn't it be neat if?"
> discussion. See the proposed extension to list comprehensions where
> "order by" was proposed from sql (a year or two ago at this point).
> That shouldn't have even made it to python-dev. Yet it did. Why?
> Because everyone has their "I really want X", and too few people are
> willing to acknowledge that they can be wrong.
Right, but I am suggesting that perhaps at least some of this problem
(the need users feel for extensions) is currently built into the
language. I also made this point to bolster my argument that
feature-creep is inevitable.
Outside of the object unification that's been happening over the last
few years, most of the features being added to Python appear to be
oriented to supporting the FP paradigm (generators, list comps, ternary
if, etc). However these features appear to be added in an ad hoc
fashion. I'm not suggesting they weren't thought out, only that they
were added based on demand rather than as part of an overall plan as the
object unification was.
It appears to me that most of these "water-cooler" suggestions are
proposed precisely because
a) Users encounter a situation where an elegant solution isn't obvious
and they feel they must resort to magic and hackery.
b) The elegant solution they were looking for is embodied as an
extension or requires knowledge that is more advanced than the problem
at hand would seem to require (e.g. metaclasses).
c) The PEPs (or more commonly, the flamewars on c.l.py) begin.
For example, earlier in this thread you provided an elegant solution to
what appeared (to myself and at least one other person) to be an
intractable problem. But of course, it wasn't the natural solution to
most people. It's ironic, because I've actually used a similar solution
before, but wasn't able to see it this time. In fact, every time I have
to use it, I have to reinvent it because it isn't the natural way the
problem is expressed. The problem is most naturally expressed
procedurally but must be solved with an OO paradigm.
> > 5) Given the added desire to maintain backwards-compatibility, old
> > features will not be shed as fast as new ones are added (unless that
> > becomes part of the process, which doesn't seem practicable to me).
>
> Python 3.0 is one of the ways that features are shed. While Guido has
> stated that he doesn't believe that there will be a Python 4k (aka
> Python 4.0) that hits the reset button again, I think that there needs
> to be one, so that Python *can* get rid of bad features that we were
> wrong about. There's also the PEP process, which as long as there
> isn't sweeping "break everything" changes, it would seem that we can
> remove features as time goes on.
Sure, and I think this is a valuable process.
> > 6) I believe that a large class of these features could be rendered moot
> > or redundant if the language embraced a more sweeping and fundamental
> > change. While this won't absolutely prevent the language's growth, it
> > provides a built-in deterrent.
>
> ...To a functional programming paradigm. That's what you meant to say, right?
Yes, although by "to" I would mean "allow for" rather than "convert to".
In other words, not convert to or even encourage the FP paradigm, but
also not prevent it. It would be embracing multi-paradigm approaches
rather than abandoning one in favor of another. Python has always been
multi-paradigm to an extent (procedural and object), but has always been
somewhat adverse to the functional paradigm (aside from a few helpful
things).
Before Python's object unification efforts, many OO purists declaimed
Python because its hybrid approach. Pythonistas claimed that they were
wrong because "practicality beats purity", etc. But of course, we've
all seen that to a large degree they were right. Practicality might
beat purity, but practicality plus purity trumps both.
Functional programming is, in fact, simply the logical extension of
procedural programming (or more precisely, procedural is a subset of
functional). They are in no way mutually exclusive. Much as Python's
OO was hampered by distinctions between builtin classes and user-defined
classes (amongst other things), so is Python's procedural support
hampered by it's rejection of the functional paradigm.
> > There's an old quote by Larry Wall: "Perl is ugly because people wanted
> > it that way". Perl took the same approach to limiting features that
> > Python does, with the notable difference that Larry didn't reject ideas
> > as consistently as Guido (or apparently at all). Nevertheless, we're
> > now simply discussing the *rate* at which the language becomes large and
> > inconsistent rather than whether it will or not.
> >
> > If we agree that the issues I outline above are valid, then I think we
> > can start bickering over possible solutions and how those solutions
> > would affect Python on the whole.
>
> I don't agree.
Well, that's a start.
Regards,
Cliff
Adam Olsen
> A slow boat still reaches its destination. My position is that by
> obviating the need for a large class of syntax extensions, the onus is
> moved from the Python core team onto the user desiring a particular
> extension and by doing so, things like community pressure to add X can
> be reduced and by extension, the likelihood of the process folding under
> that pressure becomes much less likely.
Again, a vast difference in what we consider complexity.
For a user, does it matter that we use "len(foo)" rather than "len
foo"? A little bit, but it's not huge.
For the language developers, does it matter? Again, it's a small
benefit. It's still our job to design it, not a third-party. Syntax
and stdlib are both part of the language.
All the things you think we might avoid by being more "functional",
we'd still have to do them, saving nothing. Except all the details
suggest *worse* solutions would be found, as we wouldn't be
fine-tuning the syntax for the use-cases.
The only genuine use-case for your changes I've seen in this whole
thread is the dispatch-dict, but your changes are far too distract to
justify it.
As an aside, I can't speak for Guido, but I haven't seen any
indication python's development process may collapse. It's a
non-problem.
--
Adam Olsen, aka Rhamphoryncus
Cliff Wells
> > 2) The primary method of limiting Python's core feature growth is for
> > the BDFL to dig in his heels and reject ideas until they are adequately
> > shown to be broadly needed and implementable.
>
> I don't think you have accurately described things, but it's
> irrelevant because changing this doesn't change the nature of the
> problem.
It may be grossly oversimplified, but I think ultimately the process
ends this way.
> > 3) This method has failed in the past*. In fact, I assert that this
> > method is guaranteed to fail unless all ideas that add syntactical
> > structures are rejected outright, regardless of their utility. It
> > cannot limit the growth of Python's core, it can only limit the rate at
> > which it grows.
>
> I think you're using "fail" in two different contexts here; one for an
> individual case - in that something was added - and one in the global
> case - in that the language will continue to grow without bounds.
I'm claiming that the global case is an inevitable conclusion of
repeatedly catering to the individual case.
> We'll leave aside the first version - whether some particular feature
> was a failure or not is really a question of personal taste, and
> changeable - and look at the latter.
>
> You're right, they can only limit the growth rate. I contend that that
> isn't a failure - at least not until the language starts losing users
> because the feature count has gotten too large. Based on the success
> of languages where features are treated as valuable just by being
> features, we're still a very long way from that point.
I won't disagree that Python is still a long way from that point. I
also want absolutely to distance myself from any claim that Python will
one day become obsolete or even lose a single user because of this.
First of all because it's flamebait and secondly because I don't believe
it to be true.
What I will claim is that the growing the language in such a way is not
doing a favor to its users (even if they might think so for their
particular extension). I don't think any of us disagree with that.
Of course it's mildly ironic that I'm being told the same thing about my
proposal, but what I am trying to convey is that it's my belief that my
one modification helps eliminate the need for dozens of others.
> > 5) Given the added desire to maintain backwards-compatibility, old
> > features will not be shed as fast as new ones are added (unless that
> > becomes part of the process, which doesn't seem practicable to me).
>
> This has already become part of the process. Python 3.0 can break
> backwards compatibility, and so is shedding features - or at least
> moving them out of the core. For instance, the builtin functions that
> are now subsumed into list comprehensions are gone. It was originally
> scheduled for release late this month, but it looks like the schedule
> has slipped a couple of weeks.
Yes, but we want to keep watershed instances like these to an absolute
minimum. I think removing the redundant (and less flexible) variations
that listcomp now subsumes is a good thing. What I'm saying is "let's
do it again". Except what would be subsumed isn't just a handful of
functions, but rather a whole class of existing and future language
extensions.
> > 6) I believe that a large class of these features could be rendered moot
> > or redundant if the language embraced a more sweeping and fundamental
> > change. While this won't absolutely prevent the language's growth, it
> > provides a built-in deterrent.
>
> This may well be true. But the absolutely critical issue is that it
> not break the fundamental nature of language. I.e. - if you have to
> add delimters to replace indentation, you ain't going anywhere.
Absolutely, which is why I've never proposed it. It would fundamentally
alter the visual appearance of the language and ultimately isn't
necessary anyway.
> > There's an old quote by Larry Wall: "Perl is ugly because people wanted
> > it that way". Perl took the same approach to limiting features that
> > Python does, with the notable difference that Larry didn't reject ideas
> > as consistently as Guido (or apparently at all). Nevertheless, we're
> > now simply discussing the *rate* at which the language becomes large and
> > inconsistent rather than whether it will or not.
>
> LW used to sign books with the note "There's more than one way to do
> it". As far as I can tell, the Perl community never met a feature it
> didn't like, and they're all in Perl5 - and Perl6 is going to make CL
> look small.
>
> But you just slipped in a new word that you haven't used before -
> "inconsistent". Becoming large does not necessarily make things
> inconsistent. Most of the features added to Python don't make it
> inconsistent. Actually, the nastiest problems with some of them are
> *because* they are consistent with the rest of the language(*).
True, I was thinking more of Perl when that word popped out of my
keyboard.
However, I think that there is a fundamental *logical* inconsistency in
Python. The language consistently adheres to this inconsistency =) but
it's there nonetheless. This was my motivation for attempting to
demonstrate that the two forms of "if" currently in Python (statement
and operator) could be logically combined into one if the restrictions
on the former were lifted (see below).
> Maintaining consistency is far more important than limiting size, and
> is why most proposals are rejected. I don't feel that any of the
> recent changes have introduced inconsistencies in the language.
Ah, but I do:
if cond: block
vs
expr if cond
Here we have two forms of the *very same* logical construction. A
programmer must select one or the other based on *context*. This is my
definition of inconsistent (although it's roots lie in a deeper
inconsistency, namely the distinction between expressions and
statements).
Adam Olsen
By definition, anything more complex than a turing machine contains
these redundancies (which you call inconsistencies). Clearly, we all
use languages with a great number of them, so your notion that they're
inherently bad is disproven.
Which isn't to say they're inherently good either.. they do have a
cost, increasing the mental load of the programmer. That suggests
there's a tradeoff involved. Asking about the tradeoffs would be a
much more productive discussion.
--
Adam Olsen, aka Rhamphoryncus
Cliff Wells
> On Fri, Sep 12, 2008 at 1:29 PM, Cliff Wells <cl...@develix.com> wrote:
> > A slow boat still reaches its destination. My position is that by
> > obviating the need for a large class of syntax extensions, the onus is
> > moved from the Python core team onto the user desiring a particular
> > extension and by doing so, things like community pressure to add X can
> > be reduced and by extension, the likelihood of the process folding under
> > that pressure becomes much less likely.
>
> Again, a vast difference in what we consider complexity.
Well there's two forms of complexity to consider: language complexity
and user-program complexity. A language that's too simple (e.g.
assembler or C) can lead to an explosion of complexity when actually
used to develop a non-trivial program.
I'd actually agree that language features help reduce program
complexity. For instance even the simple case of Python lacking the
equivalent of repeat..until or do..while loop forces a tiny bit of
complexity (in the form of a couple extra lines of code and usually an
extra exit point from a loop) into a user's programs.
The complexity I'm referring to is the fact that Python's collection of
statements are rapidly forking into two forms: the original statement
form, and newer expression forms. Worse, in some cases these
expression forms exist in several sub-forms (e.g. list comps, expression
comps, generators, etc). These forms all have a single common
*logical* operation in mind (looping), but mutate into other forms
because they are too specialized. It's also worth noting that most of
them use an inverted syntax from the original statement in order to help
distinguish them.
I'm claiming that a single more flexible construct could potentially be
used in place of all of them (for-expression with yield), and further,
I'm claiming this is a case where simplifying the language would not
complicate the user's program and in many cases might help to simplify
it.
> For a user, does it matter that we use "len(foo)" rather than "len
> foo"? A little bit, but it's not huge.
Or more consistently foo.len() which we *almost* have except it's
spelled __len__ :P I agree this is a minor nit, but also rather
pointless as far as I can tell.
> For the language developers, does it matter? Again, it's a small
> benefit. It's still our job to design it, not a third-party. Syntax
> and stdlib are both part of the language.
>
> All the things you think we might avoid by being more "functional",
> we'd still have to do them, saving nothing. Except all the details
> suggest *worse* solutions would be found, as we wouldn't be
> fine-tuning the syntax for the use-cases.
>
> The only genuine use-case for your changes I've seen in this whole
> thread is the dispatch-dict, but your changes are far too distract to
> justify it.
I'm not surprised. In fact, after programming in Python myself for a
decade or so I'm surprised I was able to find any suitable example ;-)
The Sapir-Whorf hypotheses predicts this. A Lisp or Haskell programmer
would probably be much better armed than I to provide concrete examples.
Even if you don't subscribe to Sapir-Whorf, I think you could probably
agree that at least the lack of idioms for FP in Python would contribute
to our mutual inability to visualize particularly useful examples.
> As an aside, I can't speak for Guido, but I haven't seen any
> indication python's development process may collapse. It's a
> non-problem.
I didn't mean the development process may collapse, I mean the rejection
process may collapse due to pressure (and I believe it has already on
occasion).
Regards,
Cliff
Adam Olsen
At most, it's a *small* benefit to the language. It comes a tradeoff,
just like a+b rather than add(a, b), or len(x) rather than x.len(). A
little bit more syntax to learn, which increases the learning curve
(but for easy enough syntax the underlying operation is more to learn
anyway), and in exchange it makes things easier for a skilled
programmer.
You're claiming we don't have to make that tradeoff.. but you've done
*nothing* to substantiate that. Even logix only adds more complexity,
rather than removing and unifying existing syntax.
We don't care about purity. Statements vs expressions.. functional vs
imperative.. they're tools, pure and simple. We switch back and forth
as appropriate. Most of the time quietly, subtly, such that you
usually don't even notice we've done it. We like it that way.
You'll be a lot more productive if you learn to relax, concerning
yourself with use-cases, not ideals.
--
Adam Olsen, aka Rhamphoryncus
Cliff Wells
> At most, it's a *small* benefit to the language. It comes a tradeoff,
> just like a+b rather than add(a, b), or len(x) rather than x.len().
Okay, you kept providing this example and I kept brushing it off because
I didn't see where you were leading. Now I do and now I realize that
I've utterly failed to get the problem across.
Let's use your example from above: a+b vs add(a, b)
a + b is the *exact* semantical equivalent of add(a, b). They do the
same thing, they can be used in the same contexts. The + operator is
pure semantic sugar (and well-used sugar too).
Now let's imagine things aren't quite so equivalent. a+b has the same
result as add(a, b), except you can only use the + operator in one
particular context: assignment. It cannot be used say, with a comparison
operator.
So in my imaginary Python, this is legal
x = 1 + 2
but this is not
if x < y + 1: pass
and must instead be written
if x < add(y, 1): pass
Now lets make it even more of an exact comparison.
Let's say the add(x,y) function didn't even exist. You have to write it
yourself. Worse, writing it requires fairly deep knowledge of Python,
and worse yet, you sometimes must write a different add(x,y) function
depending on what context you want to use it in.
At this point we've now made your example much closer in spirit to the
problem I'm attempting to present.
> You're claiming we don't have to make that tradeoff.. but you've done
> *nothing* to substantiate that. Even logix only adds more complexity,
> rather than removing and unifying existing syntax.
And you've provided nothing to substantiate this (aside from this
assertion).
Also, if we combine statements into expressions, how does this not unify
existing syntax? Combining two into one is practically the definition
of "unify".
> We don't care about purity. Statements vs expressions.. functional vs
> imperative.. they're tools, pure and simple. We switch back and forth
> as appropriate. Most of the time quietly, subtly, such that you
> usually don't even notice we've done it. We like it that way.
Who is this "we"? People who use Python? Am I not a person who uses
Python (and for the better part of 10 years I might add). I'm not some
outsider saying "can you add X from my previously favorite language Y so
I can program the way I used to". I'm someone who, until maybe a year
ago, was just as narrow-minded about Python's shortcomings as you are
now. I mean seriously, has it occurred to you that it's possible that
you simply aren't getting my point?
> You'll be a lot more productive if you learn to relax, concerning
> yourself with use-cases, not ideals.
Thanks for the advice. Should I also read a good book, maybe consider a
massage?
Regards,
Cliff
Cliff Wells
> As I've pointed out, this has already been done (although the project
> appears dead):
>
> http://www.livelogix.net/logix/index.html
And after some research this morning, I found out why the project is
dead: the author has long since moved to Ruby (where something like
Logix is rendered moot).
Figures.
Arnaud Delobelle
On 10 Sep 2008, at 22:22, Cliff Wells wrote:
> On Wed, 2008-09-10 at 21:46 +0200, Christian Heimes wrote:
>> Cliff Wells wrote:
>>> Any thoughts on this? I'm sure it's been brought up before, but I
>>> haven't found any definitive discussions on why this rather
>>> arbitrary
>>> design decision continues to hold in the face of a general migration
>>> away from imperative languages (especially when it seems it could be
>>> changed without much backwards-compatibility issues).
>>
>> Two thoughts:
>>
>> Please elaborate how you like to change the syntax of Python.
>
> No changes. Simply lifting of a particular restriction.
>
>> I like to
>> see some concrete examples how your syntax would look like. I also
>> like
>> to know how your are planing to implement features like lazy
>> evaluation.
>> The if else ternary operator statement is evaluated lazy. The same
>> construct as expression wouldn't be lazy any more.
>
> Why not?
>
> a = (
> if a > 1 then:
> long_calculation()
> else:
> other_long_calculation()
> )
>
> Clearly only one of these blocks would be evaluated at runtime.
So what does:
a = (if False: 1)
evaluate to?
--
Arnaud
Mathias Panzenböck
>
> So what does:
>
> a = (if False: 1)
>
> evaluate to?
>
OT: I attended a lecture called "abstract machines" and we had to write a little
VM (parser, bytecode interpreter) on paper during the test. It was a expression
based language and there was a "if ... then ... else ..." AND a "if ... then
..." construct. I asked the Prof. exactly the same question you did. He said:
"Oh... well... strike that out."
He really didn't notice that this is a problem when he wrote the test. *g* :P
(Sorry, for the noise.)
-panzi
Bruce Leban
Let's use your example from above: a+b vs add(a, b)
a + b is the *exact* semantical equivalent of add(a, b). They do the
same thing, they can be used in the same contexts. The + operator is
pure semantic sugar (and well-used sugar too).
Now let's imagine things aren't quite so equivalent. a+b has the same
result as add(a, b), except you can only use the + operator in one
particular context: assignment. It cannot be used say, with a comparison
operator.
So in my imaginary Python, this is legal
x = 1 + 2
but this is not
if x < y + 1: pass
and must instead be written
if x < add(y, 1): pass
Adam Olsen
Your explanation would be true, except custom functions can be used as
either statements or expressions! It is only the existing statements
which are limited to being used as statements, and they're syntactic
sugar.
>> You're claiming we don't have to make that tradeoff.. but you've done
>> *nothing* to substantiate that. Even logix only adds more complexity,
>> rather than removing and unifying existing syntax.
>
> And you've provided nothing to substantiate this (aside from this
> assertion).
>
> Also, if we combine statements into expressions, how does this not unify
> existing syntax? Combining two into one is practically the definition
> of "unify".
Merging generator expressions, list comprehensions, and the for-loop
would be unifying. Making the for-loop also usable as an expression
is generalizing a specific feature.
I've given many examples of why that doesn't work, so I'll assuming
not seeing them was due to the terminology confusion.
--
Adam Olsen, aka Rhamphoryncus
Cliff Wells
>
>
> On Sat, Sep 13, 2008 at 9:02 AM, Cliff Wells <cl...@develix.com>
> wrote:
> Let's use your example from above: a+b vs add(a, b)
>
>
> a + b is the *exact* semantical equivalent of add(a, b). They
> do the
> same thing, they can be used in the same contexts. The +
> operator is
> pure semantic sugar (and well-used sugar too).
>
>
> First, + is *syntactic* sugar not semantic sugar. Syntactic sugar is
> something that changes the appearance but not the semantics.
Sorry, clearly typing too fast. I meant semantic the first time I used
it and syntactic the second.
> C uses semicolons and Python doesn't.
Python does, but only as a substitute for a newline.
> Is that syntactic sugar? Can I just ask C to remove the "restriction"
> that semicolons are required without defining precisely what that
> means?
Arguably you'd end up with what Python does now. I think that's
well-enough defined (C's for-loop being an exception, obviously).
> If I do, then I end up with something like Javascript's semicolon rule
> which is confusing and error-prone. No thanks.
I'm not aware of JavaScript's semicolon rule. When I write JS I use it
as a line-terminator.
> Now let's imagine things aren't quite so equivalent. a+b has
> the same
> result as add(a, b), except you can only use the + operator in
> one
> particular context: assignment. It cannot be used say, with a
> comparison
> operator.
>
> So in my imaginary Python, this is legal
>
> x = 1 + 2
>
> but this is not
>
> if x < y + 1: pass
>
> and must instead be written
>
> if x < add(y, 1): pass
>
>
> This is a strawman. You hypothesize that + would only be usable in the
> context of "assignment". But that's not the context. The context is
> expression. The semantics are the same.
I was attempting to use Adam's example (which I would have labeled a
strawman, except I don't believe it was intentional) in order to convey
the difference between what he was saying and what I was saying. If it
fails, well, it's not the example I would have chosen, but I felt it
might help Adam and I find common ground for debating on, so <shrug>.
> Here's a real example: consider a language where +, *, etc. can only
> be used in the context of augmented assignment. That is:
> a += b
> c *= d
> etc. while I have to use functional syntax everywhere else:
> x = add(a,b)
> y = mul(c,d)
> etc. Now I propose improving this language by eliminating the
> "restriction" that +, * etc. can only be used in assignments.
Actually, this is the strawman. If you read back (yes, I'm asking a
lot) you'll note that I asserted that statements are (or could be)
semantically equivalent to expressions except they have a limited
context. Any statement can be made into an expression (but not the
reverse). T
> You ask me to define the syntax and semantics of these operators and I
> brush that off as unimportant. All I'm advocating is "lifting a
> restriction" in the language. I'm unifying syntax not adding
> complexity, right? No need to explain pesky details.
>
>
> No. That's ridiculous. You might be surprised when you use my new
> features that
> x = 2 * 3 + 4
> y = 2 - 3 + 4
> sets x to 14 and y to -5 but that's not my fault. (And lest you say
> this is a strawman and no one would consider any value other than 10
> and 3, go look at http://tinyurl.com/27ttcy.)
It's a matter of defining ambiguity. APL does it by being strictly
left-to-right and Python (and most other languages) defines it by a
(admittedly more complicated) algebraic order of operations.
> This is what you're doing. If you can't precisely define how the
> language is different with your proposed change, then it's not worth
> discussing. You can do that with a formal definition as I suggested or
> by actually implementing your changes as Guido suggested.
I think providing examples is enough for a general discussion of the
idea. Further, if anyone were really curious beyond just discussing it
in the abstract, they could try Logix (although you'll unfortunately
need to have Python 2.4 to do so).
I understand that any such change would need to be adequately defined.
But I don't consider the discussion to have progressed to that point.
If people do not even understand that expression-oriented languages
provide any advantage (in fact, many of them apparently consider it a
disadvantage), then there's little point in discussing what particular
syntax this apparently useless change would take on.
> You can't do it by insisting that no one understands what you're
> talking about.
When someone suggests that the distinction between expressions and
statements is syntactic sugar on the same level as the + operator, then
I have no choice but to assume they really don't understand what I'm
talking about. Whether that's a failing on my part or theirs is an open
question, but there's no doubt about the lack of understanding.
Anyway, to be frank, once I discovered that the developer of Logix had
moved to Ruby, I finally admitted to myself that I was being completely
unreasonable in expecting Python to be any more than what it is. I've
already made up my mind to move to a language I find more suitable.
Cliff Wells
That's a good question. This is one of those areas where a definition
would need to be created. My inclination is to say None (much like a
function with no return statement).
Cliff
Cesare Di Mauro
> I understand that any such change would need to be adequately defined.
> But I don't consider the discussion to have progressed to that point.
> If people do not even understand that expression-oriented languages
> provide any advantage (in fact, many of them apparently consider it a
> disadvantage), then there's little point in discussing what particular
> syntax this apparently useless change would take on.
Are you sure that calculating and returning a value for each statement
that is interpreted as an expression is really an advantage? I find little
cases that shows such an advantage on real world code (one of each is the
ternary operator, that was introduced in Python 2.5 to fill that "hole").
Just to be clear, I think that returning a value after executing a for,
while, and even an if statement/expression EVERY TIME would be of no
pratical use except for very rare cases.
Consider that you have to spend resources (CPU and memory) calculating
values, that most of the time will be trashed because they will not be
used.
I know that we already have functions that work the same way: they always
return something, defaulting to None when no return statement is used.
That's because Python has only a "function" type (contrary to Pascal,
which distinguish between functions and procedures), but I think that
Guido opted to reduce the complexity of the language giving just one
subroutine invocation type to use.
Do we really need to slow down the language (at the cost of less
readability too, because I'm strongly convinced that using statements like
expressions will reduce it) for such limited "added value"?
My 2 cents.
Cesare
Adam Olsen
That much we can agree on.
Throughout this thread very little if any progress has been made. I'm
willing to continue on IRC if you'd like, but I won't clutter the list
any more.
--
Adam Olsen, aka Rhamphoryncus
Cliff Wells
> > On Sat, 2008-09-13 at 23:58, Cliff Wells wrote:
>
> > I understand that any such change would need to be adequately defined.
> > But I don't consider the discussion to have progressed to that point.
> > If people do not even understand that expression-oriented languages
> > provide any advantage (in fact, many of them apparently consider it a
> > disadvantage), then there's little point in discussing what particular
> > syntax this apparently useless change would take on.
>
> Are you sure that calculating and returning a value for each statement
> that is interpreted as an expression is really an advantage? I find little
> cases that shows such an advantage on real world code (one of each is the
> ternary operator, that was introduced in Python 2.5 to fill that "hole").
The particular use-case I personally have in mind is for the development
of DSL's, where the domain being modeled is most naturally expressed as
nested expressions. There are many other uses and frankly many of the
things we currently use objects for could arguably be more naturally
expressed functionally.
Honestly, I think it's very difficult for a person who's grown
accustomed to a procedural style to grasp the potential here (and I
certainly include myself in this group, although I feel I'm starting to
see the light).
> Just to be clear, I think that returning a value after executing a for,
> while, and even an if statement/expression EVERY TIME would be of no
> pratical use except for very rare cases.
I remember this wisdom being encoded in the old adage "Lisp programmers
know the value of everything and the cost of nothing" which was a jab at
the relative slowness of Lisp compared to C. However the same argument
could be made for almost any aspect of a dynamic interpreted language.
I don't have any supporting data, but I strongly suspect the cost would
be almost immeasurable compared with the cost we've already incurred to
have the features we consider indispensable.
Also, it's worth noting that in the majority (actually, I'd expect
nearly all) of cases the value is already calculated, it simply isn't
returned. I think that overhead would be minimal.
> Consider that you have to spend resources (CPU and memory) calculating
> values, that most of the time will be trashed because they will not be
> used.
>
> I know that we already have functions that work the same way: they always
> return something, defaulting to None when no return statement is used.
> That's because Python has only a "function" type (contrary to Pascal,
> which distinguish between functions and procedures), but I think that
> Guido opted to reduce the complexity of the language giving just one
> subroutine invocation type to use.
I believe that was a good choice. Speaking of Pascal (which was the
second language I learned), I think there's a good parallel to be drawn.
When I was learning pointers in Pascal, I struggled badly. I assumed it
was because pointer manipulation was difficult. Then I learned C, and
discovered that in that language, pointers were so natural that I picked
up in a day what I couldn't put together properly in weeks or months in
Pascal. Pascal wanted to abstract away the machine and how data is
stored, which is anathema to pointers. C, on the other hand presented
data in a way that made pointers natural.
On the other hand, when I started doing OO programming in C (yes, it can
be done, to a limited degree), I was again in the same boat. I was able
to make it work, since I was much more expert in C than I ever was in
Pascal, but overall the experience made me realize that OO support in a
language is a good thing ;-) You can fake OO with structs and pointers
but it feels like a hack, even though it's kind of a satisfying one.
Today, you can do a fair amount of FP in Python, but it feels unnatural.
Even if you get your code to work as you want it probably won't appear
elegant. You can fake some FP constructs with OO but again, it's really
only a workaround, no matter how clever it might make you feel (in fact,
I'd go so far as to suggest that if your code makes you feel clever,
then you've just performed a workaround).
> Do we really need to slow down the language (at the cost of less
> readability too, because I'm strongly convinced that using statements like
> expressions will reduce it) for such limited "added value"?
I think reduced readability is perhaps the strongest argument against FP
style. However, even OO presents this danger and yet we embrace it
fully. I wish I could remember who this quote is attributed to (and the
exact phrasing for that matter):
"The danger of the fully realized object is that it appears ordinary"
OO presents a very arguable danger of making code impenetrable and yet
we rarely experience this problem in real life, and in fact find quite
the opposite: many things are most naturally expressed in an object
paradigm. Given that expression-oriented languages haven't collapsed
in large-scale applications (in fact they have flourished) should be
fair indication this perceived danger doesn't necessarily have to be
reality in this case either.
Overall, I think a language should trust the programmer to know what he
wants to do in his own program and that he's the one who's best able to
decide what paradigm best suits the domain at hand.
Regards,
Cliff
Cliff Wells
> On Sat, Sep 13, 2008 at 3:58 PM, Cliff Wells <cl...@develix.com> wrote:
> > On Sat, 2008-09-13 at 12:27 -0700, Bruce Leban wrote:
> >> You can't do it by insisting that no one understands what you're
> >> talking about.
> >
> > When someone suggests that the distinction between expressions and
> > statements is syntactic sugar on the same level as the + operator, then
> > I have no choice but to assume they really don't understand what I'm
> > talking about. Whether that's a failing on my part or theirs is an open
> > question, but there's no doubt about the lack of understanding.
>
> That much we can agree on.
>
> Throughout this thread very little if any progress has been made. I'm
> willing to continue on IRC if you'd like, but I won't clutter the list
> any more.
Fair enough. I'm actually willing to let it go at this point. If
someone has a point to make or a question, I won't ignore them, but I'll
refrain from indulging myself any further.
And seriously, sincere thanks for the debate. As I mentioned early on,
I didn't really expect this to go anywhere productive (except to the
extent it let me get it off my chest), so I appreciate everyone's
patience.
Regards,
Cliff
Arnaud Delobelle
Assuming the return value of "None", I go back to an example I gave
earlier:
factors = for x in range(2, n):
if n % x == 0:
x
This doesn't work as intended (filtering out the non-factors). How to
make it work? The only way I can think of is to make (if 0: 1) return
a special "non-value" which loops will then filter out. But then we
all know what happens to non-values.
So how would you solve this problem?
> Cliff
--
Arnaud
Cliff Wells
> On 13 Sep 2008, at 23:17, Cliff Wells wrote:
>
> > On Sat, 2008-09-13 at 19:01 +0100, Arnaud Delobelle wrote:
> >>
> >> So what does:
> >>
> >> a = (if False: 1)
> >>
> >> evaluate to?
> >
> > That's a good question. This is one of those areas where a definition
> > would need to be created. My inclination is to say None (much like a
> > function with no return statement).
> >
>
> Assuming the return value of "None", I go back to an example I gave
> earlier:
>
> factors = for x in range(2, n):
> if n % x == 0:
> x
>
> This doesn't work as intended (filtering out the non-factors). How to
> make it work? The only way I can think of is to make (if 0: 1) return
> a special "non-value" which loops will then filter out. But then we
> all know what happens to non-values.
>
> So how would you solve this problem?
By writing it properly ;-)
factors = for x in range ( 2, n ):
if n % x == 0:
yield x
As I mentioned previously, in order to merge the concept of generator
with a for-expression would require bringing in the yield keyword, just
as it does now for generator functions.
The example you gave would evaluate to None (or perhaps an empty list or
generator - that's a detail that would take more consideration before
defining it).
Regards,
Cliff
Arnaud Delobelle
OK, but this seems to me incompatible with current Python:
def chain(I, J):
for i in I: yield i
for j in J: yield j
Currently
>>> '-'.join(chain('spam', 'eggs'))
's-p-a-m-e-g-g-s'
With your proposal, the first *expression* (for i in I: yield i) will
evaluate to something like iter(I) and then be discarded. Then the
second *expression* (for j in J: yield j) will evaluate to something
like iter(J) which will be discarded. So chain('spam', 'eggs') will
return None.
>
> Regards,
> Cliff
>
>
--
Arnaud
Cliff Wells
It seems you have me on this one. There's clearly other ways to do the
same thing, but since backwards-compatibility is a prerequisite I'll
have to concede the point.
A potential solution would to be to use a different keyword than "yield"
to separate current syntax from my proposed syntax (that is, distinguish
expression-scoped yield from function-scoped yield), and just side-step
the issue, but that seems unappealing to me.
Regards,
Cliff
Cliff Wells
Actually, a few more minutes of pondering got me a solution I don't find
abhorrent. Let yield mean what it currently does. Instead, let
"continue" accept an optional parameter (like "return"). Then your
above example continues to work and my proposed change would look like:
for i in j:
continue i
I haven't considered this too deeply, but it is at least consistent with
"return" syntax and doesn't add a new keyword.
Cliff Wells
I'm probably replying way too fast (in fact, I know I am), but I have
two thoughts on this:
1) it seems to alter the semantics of "continue" too much when
considered against current syntax, but...
2) with the new syntax, it seems not too bad because
j = range(3)
for i in j: i # evaluates to []
for i in j: continue # evaluates to []
for i in j: continue i # evaluates to [0,1,2]
Overall I'm a bit torn on the idea. Thoughts?
Cliff Wells
>
> I'm probably replying way too fast (in fact, I know I am), but I have
> two thoughts on this:
>
> 1) it seems to alter the semantics of "continue" too much when
> considered against current syntax, but...
>
> 2) with the new syntax, it seems not too bad because
>
> j = range(3)
> for i in j: i # evaluates to []
> for i in j: continue # evaluates to []
> for i in j: continue i # evaluates to [0,1,2]
Bah, I knew I was replying too fast. I'm thinking that "continue" would
be redefined to mean "yield value and continue" which means that
for i in j: continue # evaluates to [ None, None, None ] not []
would seem the most consistent, but I fear it might be less practical
(as it would create problems trying to use for/continue inside other
expressions, although the effect when for/continue is used as a
statement remains fine).
Cliff
Arnaud Delobelle
Let's not call it continue, but YIELD for now:
for i in J: YIELD i
Now this won't work for nested loops. E.g. in current python
def flatten(I):
for J in I:
for j in J:
yield j
>>> '-'.join(flatten(['spam', 'eggs']))
's-p-a-m-e-g-g-s'
Now say you want to write that inline with a for-expression:
'-'.join(
for J in I:
for j in J:
YIELD j
)
That won't work because the j's will be accumulated in the inner loop
and the outer loop won't accumulate anything, therefore returning an
empty iterable. The flatten() example above works because the the
scope of the yield statement is clearly defined by the enclosing def
statement. To make it work, not only you need a special yield
expression but you also need a special for expression:
'-'.join(
FOR J in I:
for j in J:
YIELD j
)
Here it is clear what happens: the YIELD accumulates values in the FOR
loop. Not very coder friendly though :)
Now compare with the current syntax:
'-'.join(j for J in I for j in J)
> Regards,
> Cliff
>
--
Arnaud
Cliff Wells
> On Sun, 2008-09-14 at 01:44 -0700, Cliff Wells wrote:
>
> >
> > I'm probably replying way too fast (in fact, I know I am), but I have
> > two thoughts on this:
> >
> > 1) it seems to alter the semantics of "continue" too much when
> > considered against current syntax, but...
> >
> > 2) with the new syntax, it seems not too bad because
> >
> > j = range(3)
> > for i in j: i # evaluates to []
> > for i in j: continue # evaluates to []
> > for i in j: continue i # evaluates to [0,1,2]
>
> Bah, I knew I was replying too fast. I'm thinking that "continue" would
> be redefined to mean "yield value and continue" which means that
>
> for i in j: continue # evaluates to [ None, None, None ] not []
>
> would seem the most consistent, but I fear it might be less practical
> (as it would create problems trying to use for/continue inside other
> expressions, although the effect when for/continue is used as a
> statement remains fine).
It would *have* to evaluate to an empty list otherwise this code:
for i in range(10000000):
continue
would create a huge list as a side-effect.
So the question is, does this seem too inconsistent? Clearly returning
[None, None, None] fits nicely with how yield currently works but it's
not going to work in this case.
Cliff Wells
> On 14 Sep 2008, at 09:44, Cliff Wells wrote:
> >
> > j = range(3)
> > for i in j: i # evaluates to []
> > for i in j: continue # evaluates to []
> > for i in j: continue i # evaluates to [0,1,2]
> >
> Let's not call it continue, but YIELD for now:
>
> for i in J: YIELD i
>
> Now this won't work for nested loops. E.g. in current python
>
> def flatten(I):
> for J in I:
> for j in J:
> yield j
>
> >>> '-'.join(flatten(['spam', 'eggs']))
> 's-p-a-m-e-g-g-s'
>
> Now say you want to write that inline with a for-expression:
>
> '-'.join(
> for J in I:
> for j in J:
> YIELD j
> )
>
> That won't work because the j's will be accumulated in the inner loop
> and the outer loop won't accumulate anything, therefore returning an
> empty iterable.
How about this way instead (since for-loop is now an expression):
'-'.join(
for j in ( for J in I: YIELD J ): YIELD j
)
> Now compare with the current syntax:
>
> '-'.join(j for J in I for j in J)
Certainly more clear and concise, but since (luckily for me this time)
we're maintaining backwards-compatibility, that form would still be
available.
Cliff
Arnaud Delobelle
On 14 Sep 2008, at 10:25, Cliff Wells wrote:
>> Now say you want to write that inline with a for-expression:
>>
>> '-'.join(
>> for J in I:
>> for j in J:
>> YIELD j
>> )
>>
>> That won't work because the j's will be accumulated in the inner loop
>> and the outer loop won't accumulate anything, therefore returning an
>> empty iterable.
>
> How about this way instead (since for-loop is now an expression):
>
> '-'.join(
> for j in ( for J in I: YIELD J ): YIELD j
> )
After you've had a good night's sleep and when you look at this again,
you'll definitely think that you were too hasty in replying :)
--
Arnaud
Josiah Carlson
<arn...@googlemail.com> wrote:
>
> On 14 Sep 2008, at 10:25, Cliff Wells wrote:
>
>>> Now say you want to write that inline with a for-expression:
>>>
>>> '-'.join(
>>> for J in I:
>>> for j in J:
>>> YIELD j
>>> )
>>>
>>> That won't work because the j's will be accumulated in the inner loop
>>> and the outer loop won't accumulate anything, therefore returning an
>>> empty iterable.
>>
>> How about this way instead (since for-loop is now an expression):
>>
>> '-'.join(
>> for j in ( for J in I: YIELD J ): YIELD j
>> )
>
> After you've had a good night's sleep and when you look at this again,
> you'll definitely think that you were too hasty in replying :)
Agreed. For all of the semantic and syntactic gymnastics and
discussion about how statements -> expressions would make Python a
better language, all I can conclude from the above is "I'm glad Python
doesn't do that."
- Josiah
Cliff Wells
> On 14 Sep 2008, at 10:25, Cliff Wells wrote:
>
> >> Now say you want to write that inline with a for-expression:
> >>
> >> '-'.join(
> >> for J in I:
> >> for j in J:
> >> YIELD j
> >> )
> >>
> >> That won't work because the j's will be accumulated in the inner loop
> >> and the outer loop won't accumulate anything, therefore returning an
> >> empty iterable.
> >
> > How about this way instead (since for-loop is now an expression):
> >
> > '-'.join(
> > for j in ( for J in I: YIELD J ): YIELD j
> > )
>
> After you've had a good night's sleep and when you look at this again,
> you'll definitely think that you were too hasty in replying :)
>
Ha! I knew I should have gone to bed earlier =)
I = [ 'spam', 'eggs' ]
'-'.join (
for J in I: YIELD ( for j in J: YIELD j )
)
Now if only I'd followed Guido's suggestion I'd actually be able to test
before I post ;-)
Cliff
Cliff Wells
> Agreed. For all of the semantic and syntactic gymnastics and
> discussion about how statements -> expressions would make Python a
> better language, all I can conclude from the above is "I'm glad Python
> doesn't do that."
Well, realize that being able to do something doesn't make it the right
thing to do in a particular situation. I don't think Arnaud's goal
here is to show that functional programming is bad, rather he's forcing
me to work out whether or not it could be done without breaking existing
syntax (and a fine job he's doing too). The examples we've been
working through are testing specific cases. It doesn't mean it would be
the recommended idiom for these cases.
When I was testing the macro feature for Breve (a bad name for the
feature, I now realize), I wrote intentionally horrific code, simply to
test that the solution was general enough:
http://breve.twisty-industries.com/snippets/macro-madness
Would I ever do that in real life? Only for testing =)
Cliff
Cliff Wells
> On Sun, 2008-09-14 at 12:36 -0700, Josiah Carlson wrote:
>
> > Agreed. For all of the semantic and syntactic gymnastics and
> > discussion about how statements -> expressions would make Python a
> > better language, all I can conclude from the above is "I'm glad Python
> > doesn't do that."
Using Breve as an example again: by your reasoning, the mere existence
of Breve should be reason enough not to support OO in Python (or at the
very least, magic methods).
Breve more or less a functional DSL that barely resembles Python but
actually *is* Python (in fact, I've seen someone assert that Breve
templates could not possibly be actual Python code). Even you were
initially convinced that I was doing code-generation. Breve abuses
classes and magic methods (albeit in an entirely legal way) to create
the illusion of a declarative DSL. Clearly I think this ability is a
good thing, but it could also be argued that what I've done is
inscrutable and this type of code should not be allowed in Python.
In fact, your own example of mimicking a dispatch "table" with a class
is arguably object abuse. Personally I don't think it's a bad solution
(especially in the absence of a better way), but you've basically mapped
what is logically a functional problem onto an object. My point is that
just because something *can* be abused, it isn't reason to throw the
baby out with the proverbial bath water. The more general a tool is,
the more able it is to be used incorrectly (as anyone who's pried a lid
off a can of paint with a screwdriver can attest).
Regards,
Mathias Panzenböck
x = if cond(a):
return b
So what value is assigned to x when cond(a) is True, what value when it is
False? What value does return return? The thing is that return does definitively
not return anything, but as a statement, it can be placed everywhere in a block.
-panzi
Cliff Wells
> If I understand this right, then this would become legal too:
>
> x = if cond(a):
> return b
return would always be a syntax error outside of a function.
>
> So what value is assigned to x when cond(a) is True, what value when it is
> False?
x = if False: 1 # evaluates to None
x = if True: 1 # evalutates to True
x = if True: return 1 # syntax error: return outside of function
def foo():
x = if True: return 1 # exits the enclosing function. value of
if-expression is moot as it's never reached.
> What value does return return? The thing is that return does definitively
> not return anything, but as a statement, it can be placed everywhere in a block.
Not that it matters, but return always returns something, either a
user-specified value or None.
Regards,
Cliff
Cliff Wells
> On Sun, 2008-09-14 at 23:34 +0200, Mathias Panzenböck wrote:
> > If I understand this right, then this would become legal too:
> >
> > x = if cond(a):
> > return b
>
> return would always be a syntax error outside of a function.
>
> >
> > So what value is assigned to x when cond(a) is True, what value when it is
> > False?
>
> x = if False: 1 # evaluates to None
> x = if True: 1 # evalutates to True
I meant "evaluates to 1", of course.
Mathias Panzenböck
> Not that it matters, but return always returns something, either a
> user-specified value or None.
>
Ok, I meant it never evaluates to some value. :P
Greg Ewing
> factors = for x in range ( 2, n ):
> if n % x == 0:
> yield x
Is this construct meant to be a list comprehension or
a generator expression? If it's a GC, you just get
factors bound to an iterator, not a list of results
from that iterator. If it's an LC, then what do you
do if you want a dict or tuple comprehension instead?
Also, 'yield' already has a meaning that's different
from what you seem to want here. Consider:
def foo(n):
for x in range ( 2, n ):
if n % x == 0:
yield x
Here the whole function is a generator, and the yield
causes an item to be returned to the generator's caller.
Now, if there is no difference between a statement and
an expression, the following should do exactly the
same thing:
def foo(n):
factor = for x in range ( 2, n ):
if n % x == 0:
yield x
--
Greg
Greg Ewing
> A potential solution would to be to use a different keyword than "yield"
> to separate current syntax from my proposed syntax
But inserting any kind of keyword into LCs is going
to make them ugly and verbose. The point of having
LCs in the first place is that they express certain
kinds of things very concisely. Adding keywords and
colons to them messes that up.
Likewise with conditional expressions. It would be
disappointing if, instead of
x = a if b else c
we had to write
x = if a: b else: c
Colons in the middle of expressions look ugly, IMO.
--
Greg
Greg Ewing
> '-'.join(
> for j in ( for J in I: YIELD J ): YIELD j
> )
Noooooo..... this is getting worse and worse.
You seem to be thinking of syntax issues as though
they were purely technical puzzles. The're not --
they're at least as much human-factors issues.
> Certainly more clear and concise, but since (luckily for me this time)
> we're maintaining backwards-compatibility, that form would still be
> available.
But if you keep all the existing syntax as well,
you haven't simplified anything.
--
Greg
Cliff Wells
> Cliff Wells wrote:
>
> > factors = for x in range ( 2, n ):
> > if n % x == 0:
> > yield x
>
> Is this construct meant to be a list comprehension or
> a generator expression? If it's a GC, you just get
> factors bound to an iterator, not a list of results
> from that iterator.
First thing, I've recanted reusing "yield" (see below), but your
questions still hold in any case.
It evaluates to an iterable, I think a generator being the logical
choice (although I've been using "[]" to indicate an empty iterable in
discussion for brevity).
> If it's an LC, then what do you
> do if you want a dict or tuple comprehension instead?
Did we already grow those? I don't want to debate tuple comprehension
(I admit I fail to see the point), but dict comprehensions would be
great. To be clear, I'm aiming for as much backwards-compatibility as
possible, so I'm in no way suggesting removing comprehensions (although
I think they'd become somewhat redundant).
In the hypothetical absence of a comprehension feature, I'd probably try
to write one like these (I'm slipping "continue" in here as a
replacement for yield, which I explain below):
x = for i in j: if i: continue i else: continue i**2 # list comp
x = dict ( for i in j: continue i, i**2 ) # not a dict comp ;-)
> Also, 'yield' already has a meaning that's different
> from what you seem to want here. Consider:
Yes, that's why I had to retract using the yield keyword in this
context. Instead I propose either a new keyword or allowing continue to
accept an optional argument causing it to mean "yield expression and
continue" rather than just "continue":
factors = for x in range ( 2, n ):
in n % x == 0:
continue x
continue without an argument would continue to mean what it does now.
This raises some other concerns, but it's an option to be weighed
against adding a keyword. In any case, using "yield" is straight out.
Regards,
Cliff
Cliff Wells
> Cliff Wells wrote:
>
> > '-'.join(
> > for j in ( for J in I: YIELD J ): YIELD j
> > )
> Noooooo..... this is getting worse and worse.
>
> You seem to be thinking of syntax issues as though
> they were purely technical puzzles. The're not --
> they're at least as much human-factors issues
>
Not that it's terribly relevant to what you say, but:
'-'.join (
for J in I: YIELD ( for j in J: YIELD j )
)
is the corrected form (although YIELD continues to be a placeholder of
course).
In any case, my form does make it slightly more complicated for the
simplest case, but makes it much less complicated for more complex cases
(for the same reasons a plain for-statement can):
x = [ J [ a ]
for a in b
if c ]
vs
x = for a in b:
if c: continue J [ a ]
The complexity has barely increased and yet the second form is already
more readable (the formatting of the first reflects what I typically do
as listcomps get more complex - overkill here but it demonstrates where
I'm going).
Given that the listcomp is the only direct form of looping available as
an expression, I've written some really ugly looking ones. I've
actually taken to commenting open/close brackets simply to help
distinguish them. Not to mention, the listcomp's placement of the
yielded result before the loop and condition only works well when it's a
very simple expression. It doesn't scale well with complexity (not that
I think it was meant to).
> > Certainly more clear and concise, but since (luckily for me this time)
> > we're maintaining backwards-compatibility, that form would still be
> > available.
>
> But if you keep all the existing syntax as well,
> you haven't simplified anything.
Yes, that's unfortunate. It might, however, obviate the need for newer
ones.
Cliff
Greg Ewing
> Breve more or less a functional DSL that barely resembles Python but
> actually *is* Python (in fact, I've seen someone assert that Breve
> templates could not possibly be actual Python code).
If it looks that little like Python, I'd say it really
is a different language, and you have no right to expect
to be able to use the Python compiler as-is to process
it.
Rather than twisting Python to make it possible to
abuse it even further, you'd be better off writing a
Breve compiler in Python that produces Python code
(or maybe even translates it directly to Python
bytecode).
--
Greg
Leonardo Santagada
On Sep 14, 2008, at 10:35 PM, Greg Ewing wrote:
> Cliff Wells wrote:
>
>> Breve more or less a functional DSL that barely resembles Python but
>> actually *is* Python (in fact, I've seen someone assert that Breve
>> templates could not possibly be actual Python code).
>
> If it looks that little like Python, I'd say it really
> is a different language, and you have no right to expect
> to be able to use the Python compiler as-is to process
> it.
>
> Rather than twisting Python to make it possible to
> abuse it even further, you'd be better off writing a
> Breve compiler in Python that produces Python code
> (or maybe even translates it directly to Python
> bytecode).
I was having fun reading the discussion, specially now that it got to
the part were most people are starting to realise how ugly python
would become, to put it lightly we got to the dificult part (the "how").
But all Cliff wanted is to have better support for DSL in python, that
is his real use case. I say why don't we focus on it.
The only clean way to support DSL in python in my view would be to
support something like pyparsing or ANTLR and make it generate a
parser and some simple form of compilation to python bytecodes. How
can we support that? Is this even desired?
--
Leonardo Santagada
santagada at gmail.com
Cliff Wells
> Cliff Wells wrote:
>
> > Breve more or less a functional DSL that barely resembles Python but
> > actually *is* Python (in fact, I've seen someone assert that Breve
> > templates could not possibly be actual Python code).
>
> If it looks that little like Python, I'd say it really
> is a different language, and you have no right to expect
> to be able to use the Python compiler as-is to process
> it.
Well *I* think it looks like Python, but then I understand (as can I
think anyone familiar with Python's magic methods), but at least
superficially it gives a different impression.
In any case, Breve is mostly of interest in that it is the project that
both made me appreciate the power of expressions and start bumping into
Python's second-class support of them.
> Rather than twisting Python to make it possible to
> abuse it even further, you'd be better off writing a
> Breve compiler in Python that produces Python code
I think code generation is arguably worse than functional programming.
> (or maybe even translates it directly to Python
> bytecode).
A seriously complicated and highly technical bit of code I'd likely have
to fix every few years (and maintain multiple versions of to support
multiple Python versions)? No thanks.
In any case, I've already choked down my Python snobbery and ordered a
Ruby book. Clearly where I want to go in programming isn't available in
Python nor will it be any time soon.
Regards,
Cliff
Cliff Wells
> I was having fun reading the discussion, specially now that it got to
> the part were most people are starting to realise how ugly python
> would become, to put it lightly we got to the dificult part (the "how").
>
> But all Cliff wanted is to have better support for DSL in python, that
> is his real use case. I say why don't we focus on it.
>
> The only clean way to support DSL in python in my view would be to
> support something like pyparsing or ANTLR and make it generate a
> parser and some simple form of compilation to python bytecodes. How
> can we support that? Is this even desired?
Actually you can do quite a lot with plain objects and class magic. But
if your DSL isn't purely procedural (Breve is declarative-functional)
then you are left without much in the way of flow control as most flow
control in Python is done via statements.
Overall I've already made a decision about my programming future, but
certainly better DSL support in Python would be a good thing.
Regards,
Cliff
Greg Ewing
> Not that it's terribly relevant to what you say, but:
>
> '-'.join (
> for J in I: YIELD ( for j in J: YIELD j )
> )
No, I think you were right the first time. The above
looks like it will generate a sequence of iterators,
not a flat sequence of j values.
> x = [ J [ a ]
> for a in b
> if c ]
>
> vs
>
> x = for a in b:
> if c: continue J [ a ]
>
> The complexity has barely increased and yet the second form is already
> more readable
That's a matter of opinion. If you lay out the first one as
x = [ J [ a ]
for a in b if c ]
there's not much difference between them.
> Not to mention, the listcomp's placement of the
> yielded result before the loop and condition only works well when it's a
> very simple expression. It doesn't scale well with complexity (not that
> I think it was meant to).
You're right, it's not meant to. It's meant for the simple
cases where all the syntactic overhead of a full for-statement
and list appending code swamps the content. Given that, adding
any extra syntactic baggage, even just a 'continue' keyword,
reduces its effectiveness for its intended purpose.
Also, I think it reads quite nicely with the expression at
the beginning. It's modelled after the mathematical notation
for describing sets:
{x : some conditions on x}
--
Greg
Greg Ewing
> On Mon, 2008-09-15 at 12:37 +1200, Greg Ewing wrote:
>>If it's an LC, then what do you
>>do if you want a dict or tuple comprehension instead?
>
> Did we already grow those?
I'm not actually sure. I think Py3 has dict and set
comprehensions, but not tuple comprehensions.
--
Greg
Greg Ewing
> The only clean way to support DSL in python in my view would be to
> support something like pyparsing or ANTLR and make it generate a parser
> and some simple form of compilation to python bytecodes. How can we
> support that? Is this even desired?
The way to go is probably to have the parser build an AST.
That way you get to reuse the compiler machinery for generating
bytecode, without having to worry about the messy details
of generating textual Python source.
If the DSL->AST transformation is sufficiently context-free,
it could probably be specified using some kind of declarative
grammar.
--
Greg
Cliff Wells
> Cliff Wells wrote:
>
> > Not that it's terribly relevant to what you say, but:
> >
> > '-'.join (
> > for J in I: YIELD ( for j in J: YIELD j )
> > )
>
> No, I think you were right the first time. The above
> looks like it will generate a sequence of iterators,
> not a flat sequence of j values.
join() takes care of flattening the final yielded iterator. The first
one was actually wrong in that it didn't solve the presented problem.
> > x = [ J [ a ]
> > for a in b
> > if c ]
> >
> > vs
> >
> > x = for a in b:
> > if c: continue J [ a ]
> >
> > The complexity has barely increased and yet the second form is already
> > more readable
>
> That's a matter of opinion. If you lay out the first one as
>
> x = [ J [ a ]
> for a in b if c ]
>
> there's not much difference between them.
I thought I'd been saying that all along ;-) I think listcomps are only
a *clear* win when they are presented in their simplest form, otherwise
it really is just preference. Get into nested listcomps and the
readability (or more to the point the comprehensibility) pretty much
vaporizes.
> > Not to mention, the listcomp's placement of the
> > yielded result before the loop and condition only works well when it's a
> > very simple expression. It doesn't scale well with complexity (not that
> > I think it was meant to).
>
> You're right, it's not meant to. It's meant for the simple
> cases where all the syntactic overhead of a full for-statement
> and list appending code swamps the content. Given that, adding
> any extra syntactic baggage, even just a 'continue' keyword,
> reduces its effectiveness for its intended purpose.
So then maybe what we could agree on is that there's a place for both?
> Also, I think it reads quite nicely with the expression at
> the beginning. It's modelled after the mathematical notation
> for describing sets:
>
> {x : some conditions on x}
Sure, I think listcomps have a place. I still maintain that they are
logically redundant if you have if-expressions (not to mention less
flexible), but if we decided they were justifiable syntactic sugar then
that's fine too.
Cliff
Josiah Carlson
> On Sun, 2008-09-14 at 13:02 -0700, Cliff Wells wrote:
>> On Sun, 2008-09-14 at 12:36 -0700, Josiah Carlson wrote:
>>
>> > Agreed. For all of the semantic and syntactic gymnastics and
>> > discussion about how statements -> expressions would make Python a
>> > better language, all I can conclude from the above is "I'm glad Python
>> > doesn't do that."
> Using Breve as an example again: by your reasoning, the mere existence
> of Breve should be reason enough not to support OO in Python (or at the
> very least, magic methods).
Let me make it clear what I was expressing, because you have no idea.
I have personal opinions on what I like and dislike. It became clear
to me early on that we disagree on language aesthetics. All I was
expressing was that I'm glad that Python didn't look like what you
were writing. If you've got a problem with my expressing of that,
that's fine, you already decided to move onto a different language.
But don't claim to know what I was saying or implying about Python as
a language and what it should or should not support.
The fact is, Breve is implemented using OO in Python. Could it have
been done using a functional approach? Sure, but then you would have
had to use multi-calling semantics, closures (which is OO in
disguise), or code copy/pasting. Breve is not proof that Python
should or should not do something, and to claim otherwise (or believe
someone is claiming otherwise) is silly.
> Breve more or less a functional DSL that barely resembles Python but
> actually *is* Python (in fact, I've seen someone assert that Breve
> templates could not possibly be actual Python code). Even you were
> initially convinced that I was doing code-generation. Breve abuses
I knew that Breve was Python, but I thought it was doing something
smart by also doing transparent code generation (because there was
discussion about the "speed" of Breve) a'la Cheetah/Spitfire, because
I believed that you had worked more on the language than you actually
had. I was wrong. But that doesn't make Breve a good example of a
DSL, it makes it just another internal DSL using Python. As you say
yourself, there are many.
> classes and magic methods (albeit in an entirely legal way) to create
> the illusion of a declarative DSL. Clearly I think this ability is a
> good thing, but it could also be argued that what I've done is
> inscrutable and this type of code should not be allowed in Python.
Whom has it been argued by? I think the only argument is against
adding syntax to make Python purely functional. Using Python syntax
and semantics to write interesting features is part of what makes
Python great. I've created SQL code generators using __getattr__ and
the comparison operators plus __and__ and __or__ for where clauses.
And you know what? They looked a million times better than embedding
SQL in the code or using stored procedures. None of them would be
possible without __magic__ methods.
> In fact, your own example of mimicking a dispatch "table" with a class
> is arguably object abuse. Personally I don't think it's a bad solution
> (especially in the absence of a better way), but you've basically mapped
> what is logically a functional problem onto an object. My point is that
You didn't even understand my example. I wasn't mapping it into an
object, I was using the underlying class creation semantics
(specifically metaclasses) to generate a dispatch dictionary. I'll be
more clear this time.
def dispatcher(name, bases, dict):
import __builtin__
dispatch = {}
for name, fcn in dict.iteritems():
if callable(name):
if '_' in name:
typnam, value in name.split('_')[:2]
typ = getattr(__builtin__, typnam, None)
if not typ:
continue
dispatch[typ(value)] = fcn
else:
dispatch[name] = fcn
return dispatch
If you set the __metaclass__ of a class to the above function, you
don't get a class, you get a dictionary. A dispatch dictionary in
particular. One that works with arbitrary types (the above only works
for builtin types or those that have been inserted into the builtin
module). This method shouldn't be new to anyone. I've been doing it
since before metaclasses (using functions and locals() instead,
because I liked the look better), and I've seen it used in various
codebases. You can do similar things to build properties (I have),
and any one of a number of other really useful things.
> just because something *can* be abused, it isn't reason to throw the
> baby out with the proverbial bath water. The more general a tool is,
> the more able it is to be used incorrectly (as anyone who's pried a lid
> off a can of paint with a screwdriver can attest).
Ahh, but the example that I was complaining about *wasn't supposed to
be an abuse*, it was supposed to be an example of "how can we make
this really simple Python code work with an expression-only version of
Python?" That the *simple* Python became so complicated and ugly in
the translation to a non-existing expression-only version of Python is
unfortunate. But again, I'm glad Python doesn't look like that.
Again, don't try to tell me what my reasoning is, I was expressing an
opinion on aesthetics.
- Josiah