[Python-ideas] Syntax: 'return: ...' expressions
Yawar Amin
I've realised this is a better venue:
http://www.reddit.com/r/Python/comments/2ra8yc/return_expressions/
I have an idea for a new syntax element that combines any number of
statements and expressions into a single expression. I'm partial to
calling it a 'return' expression because that should be compatible with
existing code--there's guaranteed to be no existing code of the form
'return:'. But we could call it 'do' or 'val' or 'expr' or whatever.
expression ::= ... | return_expr
return_expr ::= "return" ":" statement* expression+
We can define all expression-oriented syntax in terms of return
expressions. I.e., wrap up all statement-oriented syntax inside an
expression. E.g.,
x = (
return:
if y == 1: z = 'a'
elif y == 2: z = 'b'
elif y == 3: z = 'c'
else: z = 'd'
z
)
Or a single-line example,
x = return: print "Blah"; 5
I know, it's the Python programmer's dreaded multi-line expression. Note
here that I'm not proposing any change to indentation rules. I'm relying
on parens to relax the rules. There's precedent for using parens in new
kinds of expressions--e.g. generator expressions. So the usage shouldn't
look alien in Python code.
Now the controversy. We can use a return expression to get a multi-line,
multi-statement, lambda, as long as lambda only cares that its body is
a single exprssion. Which I believe is the case, e.g. this is valid
Python today:
f = lambda x: (
x,
x + 1,
x + 2
)
Anyway, an imaginary lambda example:
txtName.text.signal.map(
# Need to wrap the return expr in parens to separate the lambdas.
lambda s: (
return:
l = len(s)
l >= 5
),
# The next argument, a lambda without a return expr, happily lives
# alongside.
lambda err: notifications.send(err)
).subscribe(
# Parens not needed here, only a single argument.
lambda flag:
return:
pnlMain.back_color = "green" if flag else "red"
# Must end with an expr, unlike a normal function. Think of it
# this way: we're 'inside' a return, we _have to_ return
# some value.
None
)
I believe 'return: ...' is an unintrusive and versatile solution. It's
_not_ meant to just forcefully shoehorn full functionality into lambdas
as I believe I show above; it doesn't break compatibility; it doesn't
require any indent/whitespace rule changes; and it's guaranteed to not
affect _any_ existing code.
Regards,
Yawar
P.S. Relevant:
https://groups.google.com/d/msg/python-ideas/kYQbvsmyM-4/ufU26RPTjLQJ
'A much better idea would be to find a way to make all compound
statements into expressions, future-proofing the decision and avoiding
the redundancy between "compound statement statement" and "compound
statement expression".'
https://groups.google.com/d/msg/python-ideas/EQQq3--DDu0/UTcfI34sVwwJ
'Something that has started to annoy me in the last couple of years is
the fact that most Python control statements cannot be used as
expressions. I feel this is a pretty deep limitation and personally I
don't feel it's well-justified.'
Andrew Barnert
> I know, it's the Python programmer's dreaded multi-line expression. Note
> here that I'm not proposing any change to indentation rules.
You're using multiple statements inside parentheses, indented as if they were a block. (This is effectively just a suggestion for brace syntax, and I'm not sure what the return adds to it that you couldn't get just from brace syntax. You also seem to have invented a rule that a sequence of statements has the value of the last statement as its value. Which requires first inventing a rule that gives values to statements. I'll assume you wanted to go with the same rule the interactive interpreter uses to display a last value--an expression statement has its expression's value, and any other kind of statement has None?)
You're assuming that parentheses have an existing indentation rule that you can piggyback on, but they don't. Superficially, parenthesis continuation may look similar to block indentation, but it's not at all the same. Compare:
>>> (spam,
... eggs)
>>> if spam:
... eggs
The first is a perfectly valid tuple display; the second raises an IndentationError. Because suites begin with a new indent, but paren continuation just concatenates the lines together.
So, unless you're suggesting that any free-form sequence of statements is now legal within parens, you must be inventing a new indent rule to use within those parens.
There are also problems with nesting indents and colons. Note that compound statements (like if) include either by an inline simple statement list, or an indented block of one or more statements. This means you can't have two possibly-indenting colons on the same line. Unless your new expression can only be used in simple statements, and a simple statement with your new expression can't be used in an inline simple statement list, you're changing the way colons and indents are parsed by statements.
Finally, are you sure your new return: isn't ambiguous without look ahead or context? When the parser reads "if spam: return", is it starting a return simple statement, or an expression statement that starts with a return expression? Have you tried actually writing out the grammar?
_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Chris Angelico
<abar...@yahoo.com.dmarc.invalid> wrote:
> Finally, are you sure your new return: isn't ambiguous without look ahead or context? When the parser reads "if spam: return", is it starting a return simple statement, or an expression statement that starts with a return expression? Have you tried actually writing out the grammar?
>
simple return statement would have no colon. Your other concerns,
though, are quite valid.
ChrisA
Andrew Barnert
> On Mon, Jan 5, 2015 at 8:49 PM, Andrew Barnert
> <abar...@yahoo.com.dmarc.invalid> wrote:
>> Finally, are you sure your new return: isn't ambiguous without look ahead or context? When the parser reads "if spam: return", is it starting a return simple statement, or an expression statement that starts with a return expression? Have you tried actually writing out the grammar?
>
> The way I see it, the return-expression is "return:", whereas the
> simple return statement would have no colon.
I suspect it probably _is_ possible to make something work that fits into Python's parser, but it isn't immediately obvious (at least to me) what that would be, which is why I wanted to see a grammar. (And also to know which "level" this comes at, to automatically answer questions like how it fits into things like conditional expressions and comprehension clauses and so forth.)
Chris Angelico
> Are you suggesting that "return:" would be a single token (as opposed to all the other colons in the tokenizer), or that you'd use look ahead to disambiguate?
I'm sure this can be dealt with, though; if nothing else, requiring
that this kind of expression be parenthesized would do it, as 'return'
inside parens has no meaning. It'd be like a genexp.
But I still don't think it's a good idea :)
ChrisA
Yawar Amin
> ... paren continuation just concatenates the lines together.
I'll quote this first, because it's the crux of the matter and what I
failed to understand up until now. IIUC now, Python never actually
'relaxes' any indentation rules--a preprocessor just concatenates split
lines (i.e. lines split with parens etc.) and then passes the result on
to the parser?
I guess pretty much everything else is made redundant by this one point.
> ... I'm not sure what the return adds to it that you couldn't get just
> from brace syntax.
I was trying to avoid brace syntax as un-Pythonic.
> You also seem to have invented a rule that a sequence of statements
> has the value of the last statement as its value. Which requires first
> inventing a rule that gives values to statements. I'll assume you
> wanted to go with the same rule the interactive interpreter uses to
> display a last value--an expression statement has its expression's
> value, and any other kind of statement has None?)
is evaluated, and then the last expression inside the block becomes the
value of the block as a whole. No change would be required to any
existing expressions or statements, or to the result of a normal
sequence of statements.
> Finally, are you sure your new return: isn't ambiguous without look
> ahead or context? When the parser reads "if spam: return", is it
> starting a return simple statement, or an expression statement that
> starts with a return expression? Have you tried actually writing out
> the grammar?
easily be a new one like 'do' or 'begin'.
It's back to the drawing board for me, I guess :-)
Regards,
Yawar
Andrew Barnert
> On 2015-01-05 04:49, Andrew Barnert wrote:
>
>> You also seem to have invented a rule that a sequence of statements
>> has the value of the last statement as its value. Which requires first
>> inventing a rule that gives values to statements. I'll assume you
>> wanted to go with the same rule the interactive interpreter uses to
>> display a last value--an expression statement has its expression's
>> value, and any other kind of statement has None?)
>
> No, I was trying to say that whatever's inside the 'return: ...' block
> is evaluated, and then the last expression inside the block becomes the
> value of the block as a whole. No change would be required to any
> existing expressions or statements, or to the result of a normal
> sequence of statements.
One type of statement, the expression statement, is just an expression on a line by itself (or between semicolons), so you _could_ invent a rule pretty easily that expression statements have the value of their expression, and all other statements have a value of None, and a block has the value of its last statement. This is basically the same rule used by the interactive REPL for displaying the repr of what you've typed and setting the _ variable, so it wouldn't be a huge stretch to add the same rule to the language itself--but it would still be a new rule, and a significant change to the internals of the interpreter, even though it would only be useful in this new context.
Nick Coghlan
> On 2015-01-05 04:49, Andrew Barnert wrote:
>> ... paren continuation just concatenates the lines together.
>
> I'll quote this first, because it's the crux of the matter and what I
> failed to understand up until now. IIUC now, Python never actually
> 'relaxes' any indentation rules--a preprocessor just concatenates split
> lines (i.e. lines split with parens etc.) and then passes the result on
> to the parser?
the tokeniser. They're only generated to delimit statements, never
inside expressions - by the time the parser itself gets involved, the
original whitespace has been elided by the tokenisation process.
> It's back to the drawing board for me, I guess :-)
an interesting read.
Cheers,
Nick.
--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia
Eugene Toder
<abar...@yahoo.com.dmarc.invalid> wrote:
> I think you're missing another important point here: statements and expressions are different things. Blocks are made up of statements, not expressions, and statements don't have values. This is different from many other languages like C, Ruby, or JavaScript, where as many things as possible are expressions (and therefore have values), so a block is (oversimplifying a bit) just a sequence of expressions separated by semicolons and stuck inside braces, and therefore it has a last value.
statements don't have values. This is the reason, for example, for
having the ?: operator, and for having statement expressions in GNU C
[1]. This separation is common in C-like languages. Some C-like
languages that have REPL invent a rule for statement values similar to
Python's REPL.
> One type of statement, the expression statement, is just an expression on a line by itself (or between semicolons), so you _could_ invent a rule pretty easily that expression statements have the value of their expression, and all other statements have a value of None, and a block has the value of its last statement. This is basically the same rule used by the interactive REPL for displaying the repr of what you've typed and setting the _ variable, so it wouldn't be a huge stretch to add the same rule to the language itself--but it would still be a new rule, and a significant change to the internals of the interpreter, even though it would only be useful in this new context.
idea. All that is needed is to end the "return expression" (or
whatever you want to call it) with an expression rather than a
statement (i.e. return_expr ::= "keyword" statement* expression). The
value of this last expression is the value of the whole thing.
[1] https://gcc.gnu.org/onlinedocs/gcc-3.1/gcc/Statement-Exprs.html
Yawar Amin
> [...]
> expressions are different things. Blocks are made up of statements,
Actually I can offer counter-examples to that:
def p(x): print x
class A: p("Hello!")
class B: 1
# Etc.
But I think I see what's happening here: statements are the top dogs in
Python, and when Python wants a statement but only has an expression, it
'promotes' the expression into a statement by evaluating it, throwing
away its value, and pretending nothing happened (i.e., that there was a
'pass' statement there). Thus defining a class can have the nonsensical
effect of calling a function.
But there's currently no way of going in the other direction, i.e.
demoting a statement to an expression. Which is what I was trying to do.
I _still_ think at least something like the following would work (in
terms of Python's grammar[1]):
expr_expr: 'expr' ':' small_stmt (';' small_stmt)* [';' expr]
So e.g.:
x = expr: import os; os.system("date")
I'll explore this further and see where it goes.
Regards,
Yawar
[1] https://docs.python.org/3/reference/grammar.html
Chris Angelico
> But I think I see what's happening here: statements are the top dogs in
> Python, and when Python wants a statement but only has an expression, it
> 'promotes' the expression into a statement by evaluating it, throwing
> away its value, and pretending nothing happened (i.e., that there was a
> 'pass' statement there). Thus defining a class can have the nonsensical
> effect of calling a function.
expr_stmt, which contains an expression and (if I'm not misreading the
grammar) may or may not assign it to anything.
https://docs.python.org/3/reference/grammar.html
It's not that an expression gets "promoted", it's that one valid form
of statement is an expression with nothing around it. In C, the
definition is more like "an expression followed by a semicolon", but
in Python, you don't need any adornment, so it looks identical.
However, there is (as far as I know) no way in Python to wrap a
statement into an expression. Some languages have such a thing (eg a
lambda function syntax, which is an expression returning a function,
and in which statements are allowed), but I don't think Python does,
anywhere; all compound units that allow statements are themselves
statements.
ChrisA
MRAB
> On 2015-01-06 03:25, Andrew Barnert wrote:
>> [...]
>> I think you're missing another important point here: statements and
>> expressions are different things. Blocks are made up of statements,
>> not expressions, ...
>
> Actually I can offer counter-examples to that:
>
> def p(x): print x
> class A: p("Hello!")
> class B: 1
> # Etc.
>
> But I think I see what's happening here: statements are the top dogs in
> Python, and when Python wants a statement but only has an expression, it
> 'promotes' the expression into a statement by evaluating it, throwing
> away its value, and pretending nothing happened (i.e., that there was a
> 'pass' statement there). Thus defining a class can have the nonsensical
> effect of calling a function.
>
> But there's currently no way of going in the other direction, i.e.
> demoting a statement to an expression. Which is what I was trying to do.
> I _still_ think at least something like the following would work (in
> terms of Python's grammar[1]):
>
> expr_expr: 'expr' ':' small_stmt (';' small_stmt)* [';' expr]
>
> So e.g.:
>
> x = expr: import os; os.system("date")
>
In BCPL, the body of a routine can be only a single statement, but that
statement can be a block:
LET foo() BE
$(
....
$)
Similarly, the body of a function can be only an expression, but that
expression can include VALOF. VALOF is followed by a block, and the
value of VALOF is given by the expression after RESULTIS:
LET foo() = VALOF
$(
...
RESULTIS ...
...
$)
Of course, in Python, the block would be indented, so it's not entirely
suitable, but you could imagine this:
x = valof:
import os
resultis os.system("date")
I think you're better off using a function instead!
Andrew Barnert
> On Tue, Jan 6, 2015 at 3:25 AM, Andrew Barnert
> <abar...@yahoo.com.dmarc.invalid> wrote:
>> I think you're missing another important point here: statements and expressions are different things. Blocks are made up of statements, not expressions, and statements don't have values. This is different from many other languages like C, Ruby, or JavaScript, where as many things as possible are expressions (and therefore have values), so a block is (oversimplifying a bit) just a sequence of expressions separated by semicolons and stuck inside braces, and therefore it has a last value.
> Statements and expressions are separate in C and Javascript,
In C, unlike Python (and, more to the point, unlike many of C's predecessors), assignment, augmented assignment, and increment are expressions. Different languages have since taken that farther: Ruby makes most flow control into expressions, JavaScript makes function definitions into expressions (which means you can wrap any statement in an expression by defining and calling a function around it), CoffeeScript makes everything but return, continue, and break into expressions.
>> One type of statement, the expression statement, is just an expression on a line by itself (or between semicolons), so you _could_ invent a rule pretty easily that expression statements have the value of their expression, and all other statements have a value of None, and a block has the value of its last statement. This is basically the same rule used by the interactive REPL for displaying the repr of what you've typed and setting the _ variable, so it wouldn't be a huge stretch to add the same rule to the language itself--but it would still be a new rule, and a significant change to the internals of the interpreter, even though it would only be useful in this new context.
> I don't think one needs to invent this rule in the context of this
> idea. All that is needed is to end the "return expression" (or
> whatever you want to call it) with an expression rather than a
> statement (i.e. return_expr ::= "keyword" statement* expression). The
> value of this last expression is the value of the whole thing.
I'm assuming that the OP wanted something that works intuitively similar to other languages, the REPL, etc.--e.g., if the last statement executed is in the true block of an if/else, it doesn't matter that it's not textually the last statement.
Yawar Amin
> [...]
> break, or continue, because you can theoretically write anything to
> fall off the end.
syntax doesn't need a return statement because it is defined to
automatically return its last expression (i.e., 'fall off the end'). All
other control structures and syntax keep their existing behaviour,
including functions.
And in any case, many languages don't have any concept of a return
statement; they _do_ actually return the last expression in their
function bodies.
> I'm assuming that the OP wanted something that works intuitively
> similar to other languages, the REPL, etc.--e.g., if the last
> statement executed is in the true block of an if/else, it doesn't
> matter that it's not textually the last statement.
expressions. That would unify the two concepts in Python and make it
much more expressive.
Regards,
Yawar
Nick Coghlan
> Not exactly, I think ... I just wanted a way to 'wrap' statements inside
> expressions. That would unify the two concepts in Python
by statements, are deliberately distinct from the computational
aspects, as represented by expressions.
Some constructs blur the boundaries between computation, data
modelling and control flow, and hence may exist in both statement and
expression forms.
The fact that it permits data modelling and control flow constructs to
be embedded inside computational ones is an argument *against* your
proposal, not one in its favour.
> and make it much more expressive.
inside expressions doesn't make the language more expressive overall,
it just lets you do more within a single expression without giving the
results of suboperations names and/or factoring them out into separate
usable functions.
The number of cases where executing a suboperation first and binding
the result to a name even arguably reduces clarity are relatively few
and far between, and better represented in PEPs like 403 and 3150 than
they are in a general purpose statement-as-expression syntax.
So if you're interested in pursuing this further, I suggest focusing
on the possible pragmatic benefits of defining a standard way to
tunnel Python code through whitespace insensitive contexts and
reformat it with a pretty printer at the far end, rather than the far
more nebulous concept of expressiveness.
Regards,
Andrew Barnert
> Actually I can offer counter-examples to that:
>
> def p(x): print x
> class A: p("Hello!")
> class B: 1
> # Etc.
> But I think I see what's happening here: statements are the top dogs in
> Python, and when Python wants a statement but only has an expression, it
> 'promotes' the expression into a statement by evaluating it, throwing
> away its value, and pretending nothing happened (i.e., that there was a
> 'pass' statement there). Thus defining a class can have the nonsensical
> effect of calling a function.
>
> But there's currently no way of going in the other direction, i.e.
> demoting a statement to an expression. Which is what I was trying to do.
Yawar Amin
> [...]
>
> That's the key. A function is a way to wrap a sequence of statements (the function body) so it can be used in an expression (a function call). Because JavaScript lets you define functions (with full statement syntax) in expressions, that gives you a way to "demote" an expression inline, while Python can only do so out-of-line. At the core, this is what all of the multiline lambda attempts and similar proposals are trying to accomplish.
Regards,
Yawar
Yawar Amin
> On 2015-01-07, at 2:11, Andrew Barnert <abar...@yahoo.com> wrote:
>> [...]
>> That's the key. A function is a way to wrap a sequence of statements
>> (the function body) so it can be used in an expression (a function
>> call). Because JavaScript lets you define functions (with full
>> statement syntax) in expressions, that gives you a way to "demote" an
>> expression inline, while Python can only do so out-of-line. At the
>> core, this is what all of the multiline lambda attempts and similar
>> proposals are trying to accomplish.
>
> Thanks. This has given me an idea that's on a slightly different
> tangent than this one. Will write it up (this time with some actual
> runnable code) when I get home tonight.
http://yawar.blogspot.ca/2015/01/expressive-functional-programming-with.html
I have a feeling that some Pythoneers will really hate it, but hey, what
can you do :-)
Regards,
Yawar
Chris Angelico
> As promised, a write-up of my idea:
> http://yawar.blogspot.ca/2015/01/expressive-functional-programming-with.html
>
> I have a feeling that some Pythoneers will really hate it, but hey, what
> can you do :-)
head that 'print' is a function and not a statement :)
I'm really not seeing the advantage of this system; instead of
expressing your code as a series of statements, you have to nest
everything inside everything else, and the advantage is... the purity
of having everything be an expression. Seems to me this has purpose in
penetration testing (eg proving that you can get past someone else's
sandboxing), but not practical programming.
ChrisA
Yawar Amin
> [...]
> head that 'print' is a function and not a statement :)
my shame: I'm still on 2.7.3.
> I'm really not seeing the advantage of this system; instead of
> expressing your code as a series of statements, you have to nest
> everything inside everything else, and the advantage is... the purity
> of having everything be an expression. Seems to me this has purpose in
> penetration testing (eg proving that you can get past someone else's
> sandboxing), but not practical programming.
with someone saying 'That's never going to have a practical
application', so I could take that as a good sign :-)
Regards,
Yawar
Ben Finney
> Ah, I'll have to remember that from next time onwards to continue
> hiding my shame: I'm still on 2.7.3.
If you are interested in the future of Python (which is what this forum
entails), you will be at cross purposes until you switch to Python 3.
Those who want to continue writing for Python 2 are welcome to do so,
but Python 2 is not a consideration for discussing improvements to
Python.
--
\ “A life spent making mistakes is not only most honorable but |
`\ more useful than a life spent doing nothing.” —anonymous |
_o__) |
Ben Finney
Yawar Amin
On 2015-01-07 20:47, Ben Finney wrote:
> [...]
> entails), you will be at cross purposes until you switch to Python 3.
>
> Those who want to continue writing for Python 2 are welcome to do so,
> but Python 2 is not a consideration for discussing improvements to
> Python.
never proposed anything that would only work with an older version.
The comments you're replying to are about something I posted on my own
blog, which you'll agree is not python-ideas.
Regards,
Yawar
Chris Angelico
your assumption (on the blog) that 'print' was a statement. If you use
other statements, your code will (as far as I know) work equally on
2.7 and 3.x, which is an advantage, not a disadvantage.
The content of your post is all stuff that can be done _without_
proposing changes to the language, so it's perfectly appropriate to
talk about both branches.
ChrisA
Andrew Barnert
The difference isn't just a lack of syntactic sugar: in a pure expression language, "is followed by" has to be modeled as "is controlled by" (with CPS that's made explicit), meaning (in Python) it has to be indented.
Haskell's do statement has a nice way out of that (if you really want to turn off Pythonistas by using the word "monad"). Promise-based sequencing (as with most JS Promises/A-based code) offers a different solution with a similar effect. It might be worth following one of those ideas to see where it leads you.
In fact, the latter raises another possibility: once you're yielding from promises, you've got coroutines for free. In at some cases, explicit CPS code can be turned into implicit coroutine code very easily. If that works for everything, it also means you no longer need the manual trampoline and/or you can make all of your sequencing asynchronous and therefore add in concurrency for free (as in asyncio).
Another trick that might be worth playing with explicit native globals and locals for your functions (e.g., by constructing a new types.FunctionType), so let bindings (which also take care of def, by just binding a lambda, and class, by binding a call to type or the appropriate metaclass) can use native Python environments (and closures). This would be more interesting if you wanted to add state later (because you'd get it for free), but it might be interesting just to see where you run into problems.
Anyway, while this raises some interesting ideas, I don't think it raises any ideas for the future of the Python language, so I'm not sure Python-ideas is the right place to discuss it.
Nick Coghlan
> As promised, a write-up of my idea:
> http://yawar.blogspot.ca/2015/01/expressive-functional-programming-with.html
you describe in the final article of your post. You will have zero
chance of persuading anyone if you start from an assumption that the
distinction is accidental.
That said, there are some *very* limited situations where pulling
things out-of-line in order to name them reduces readability (see PEPs
403 and 3150 for further discussion of that), and other situations
where semantically significant leading whitespace causes practical
problems (see http://python-notes.curiousefficiency.org/en/latest/pep_ideas/suite_expr.html
for the discussion of that).
If you can come up with a practical proposal that better handles those
situations, *without* having a generally negative effect on code
readability, and without introducing "two ways to do it" for various
problems that will complicate learning, then you may have something
signficant.
But for something as fundamental as introducing new syntax, you need
to start with concrete use cases, rather than hypothetical "it might
be useful if..." situations.
Andrew Barnert
> The statement/expression distinction is deliberate, for the reasons
> you describe in the final article of your post. You will have zero
> chance of persuading anyone if you start from an assumption that the
> distinction is accidental.
Many other "modern" languages (Ruby, CoffeeScript, etc.) try to eliminate or hide the distinction as much as possible. And JavaScript documentation often apologizes for having the distinction and explains how you can "work around the problem" by defining and calling a function inline to wrap any statement. So, many people believe that the distinction is all negatives with no positives.
I believe the distinction isn't just there to keep the parser simpler, or because Guido wasn't clever enough to figure out how to embed statements in expressions in an indentation-significant language, or because the examples of Ruby and CoffeeScript didn't exist yet and now it's too late for Python; statements are a big part of what makes Python code more readable than those languages, both in-depth and in a quick scan to see the general flow control and state changes.
But it's hard to explain why. And, even if I manage to explain why _I_ think statements make Python more readable, that's just my opinion; it would be nice to have something that Guido endorsed.
Also, for people who want to suggest changes to Python (or design their own languages), it would be helpful to understand how the distinction fits in with other things. For example, in a pure functional language, there may be no benefit to having statements. In a language where (almost) everything is an expression, having mutating methods return self instead of None wouldn't be nearly as much of a problem. More generally, expression-ness and not-mutating are deeply connected (e.g., comprehensions aren't designed for mutation, loop statements are), but it's not easy to say how. And so on.
Nick Coghlan
people that they look at a cookbook written in English, and then try
translating that cookbook into their preferred programming language in
such a way that anyone who could read the original English version
could still read the translated version without needing to be told
"Oh, don't worry about that bit, it's just needed so the computer
understands what is going on").
When a problem is amenable to being handled in that style, you're
often best off coding it in that style, as that opens up future
maintenance to the broadest possible base of contributors. Languages
that support coding in this style are the ones typically referred to
as "scripting languages", and Python is one of those languages.
However, there are additional styles of thinking that folks can be
trained in (like lambda calculus, object oriented programming, event
driven programming, etc) and the reason those styles of thinking were
created is because the cookbook style starts to encounter scalability
problems when trying to handle higher levels of complexity.
Many programming languages are written on the assumption that a single
style of thinking should be applied to the entire problem space of an
application, and then get referred to by the style of thinking they
promote (so functional programming languages, object-oriented
programming languages, logical assertion languages, etc). This
generally limits the applicability of those languages outside the
problem domains where that particular style of thinking is suitable,
and also means their barriers to entry are much higher than those of
the scripting languages (since you need to learn a new way of thinking
first, whereas scripting languages often let you talk to the computer
using the same kind of reasoning that you'd use to give detailed
step-by-step instructions to another human).
The design of Python takes a more pragmatic approach and recognises
that different problems and situations are best handled through
different kinds of thinking, and so it offers those features as
optional add-ons to the cookbook approach, allowing people to use them
if they make sense in their particular case, *without* making them the
dominant mode of operation of the overall language.
Where folks run into trouble with Python (especially when it comes to
the design side of things) is in trying to ignore that overarching
cookbook element, and *just* use one of the other styles of thinking.
It's not designed to work that way - the "cookbook" layer defines the
overall structure, and then the other aspects like the object-oriented
programming support, the functional programming support and the event
driven programming support all reside within that as suitable tools
for building *components* of larger systems.
Yawar Amin
On 2015-01-08 10:32, Andrew Barnert wrote:
> [...]
> Another trick that might be worth playing with explicit native globals
> and locals for your functions (e.g., by constructing a new
Thanks for the ideas. I have to admit I didn't really think deeply about
most of them, but the above two resonated and gave me a couple of ideas
myself. I was able to refine my continuations idea, using monadic
sequencing of effects and defining a new function-like type, so that at
this point I can do this:
actions = {
"hello":
print_("Hello, World!", λ:
import_("math", λ m:
print_("The area of my circle is:", λ:
print_(m.pi * 5 * 5)))),
"goodbye":
print_("Goodbye, Cruel World!", λ:
try_(λ:
1 / 0, λ:
print_("Danger, Will Robinson!"))) }
actions["hello"]()
actions["goodbye"]()
For now, I'm satisfied that this is the best I can do towards resolving
the duality of statements and expressions. As you and others have
pointed out, this duality is intentional and is meant to guide the
programmer into a certain mindset, which is why fighting against it is
so hard.
But often when you're just trying to express the ideas in your head,
sometimes you just _have_ to get something out in a certain way. So I
think there's utility in my approach.
As you say, this isn't really a relevant 'Python idea' for this mailing
list, so this discussion isn't really going to go anywhere. Anyway,
thanks for your input, it's appreciated.
Regards,
Yawar
Franklin? Lee
<abar...@yahoo.com.dmarc.invalid> wrote:
>
> I believe the distinction isn't just there to keep the parser simpler, or because Guido wasn't clever enough to figure out how to embed statements in expressions in an indentation-significant language, or because the examples of Ruby and CoffeeScript didn't exist yet and now it's too late for Python; statements are a big part of what makes Python code more readable than those languages, both in-depth and in a quick scan to see the general flow control and state changes.
>
> But it's hard to explain why. And, even if I manage to explain why _I_ think statements make Python more readable, that's just my opinion; it would be nice to have something that Guido endorsed.
>
> Also, for people who want to suggest changes to Python (or design their own languages), it would be helpful to understand how the distinction fits in with other things. For example, in a pure functional language, there may be no benefit to having statements. In a language where (almost) everything is an expression, having mutating methods return self instead of None wouldn't be nearly as much of a problem. More generally, expression-ness and not-mutating are deeply connected (e.g., comprehensions aren't designed for mutation, loop statements are), but it's not easy to say how. And so on.
Will this Guido post suffice as a substitute?
http://www.artima.com/weblogs/viewpost.jsp?thread=147358
Yawar might also need this, if he chooses to pursue this path.
https://wiki.python.org/moin/AlternateLambdaSyntax
Andrew Barnert
> On Fri, Jan 9, 2015 at 3:46 AM, Andrew Barnert
> <abar...@yahoo.com.dmarc.invalid> wrote:
>>
>> I believe the distinction isn't just there to keep the parser simpler, or because Guido wasn't clever enough to figure out how to embed statements in expressions in an indentation-significant language, or because the examples of Ruby and CoffeeScript didn't exist yet and now it's too late for Python; statements are a big part of what makes Python code more readable than those languages, both in-depth and in a quick scan to see the general flow control and state changes.
>>
>> But it's hard to explain why. And, even if I manage to explain why _I_ think statements make Python more readable, that's just my opinion; it would be nice to have something that Guido endorsed.
>>
>> Also, for people who want to suggest changes to Python (or design their own languages), it would be helpful to understand how the distinction fits in with other things. For example, in a pure functional language, there may be no benefit to having statements. In a language where (almost) everything is an expression, having mutating methods return self instead of None wouldn't be nearly as much of a problem. More generally, expression-ness and not-mutating are deeply connected (e.g., comprehensions aren't designed for mutation, loop statements are), but it's not easy to say how. And so on.
>
> Sorry to be late.
>
> Will this Guido post suffice as a substitute?
> http://www.artima.com/weblogs/viewpost.jsp?thread=147358
But I still think Python gains more from the inescapable statement-expression than this. Guido's answer is sufficient in itself, but there's more there even if it weren't true.
I suppose the way to answer that would be to come up with a language where complex expressions are also indentation-sensitive (and there's no paren or backslash continuation) and show that if you get rid of the stack of stateful modes, there really is no other problem putting statements inside expressions. That's kind of the opposite of the way people usually try to tackle this problem, but if Guido's right that the usual way is inherently bound to fail...
Guido van Rossum
On Jan 20, 2015, at 9:15, "Franklin? Lee" <leewangzh...@gmail.com> wrote:
> On Fri, Jan 9, 2015 at 3:46 AM, Andrew Barnert
> <abar...@yahoo.com.dmarc.invalid> wrote:
>>
>> I believe the distinction isn't just there to keep the parser simpler, or because Guido wasn't clever enough to figure out how to embed statements in expressions in an indentation-significant language, or because the examples of Ruby and CoffeeScript didn't exist yet and now it's too late for Python; statements are a big part of what makes Python code more readable than those languages, both in-depth and in a quick scan to see the general flow control and state changes.
>>
>> But it's hard to explain why. And, even if I manage to explain why _I_ think statements make Python more readable, that's just my opinion; it would be nice to have something that Guido endorsed.
>>
>> Also, for people who want to suggest changes to Python (or design their own languages), it would be helpful to understand how the distinction fits in with other things. For example, in a pure functional language, there may be no benefit to having statements. In a language where (almost) everything is an expression, having mutating methods return self instead of None wouldn't be nearly as much of a problem. More generally, expression-ness and not-mutating are deeply connected (e.g., comprehensions aren't designed for mutation, loop statements are), but it's not easy to say how. And so on.
>
> Sorry to be late.
>
> Will this Guido post suffice as a substitute?
> http://www.artima.com/weblogs/viewpost.jsp?thread=147358
That's not what I was looking for--but it is a great counter to my dismissal of "just to keep the parser simpler". Guido is of course right. I wasn't thinking about the fact that statement parsing and expression parsing are different modes, with state, and it's a lot harder for a human mind to subconsciously keep a stack of stateful modes in short-term memory than for a program.
But I still think Python gains more from the inescapable statement-expression than this. Guido's answer is sufficient in itself, but there's more there even if it weren't true.
I suppose the way to answer that would be to come up with a language where complex expressions are also indentation-sensitive (and there's no paren or backslash continuation) and show that if you get rid of the stack of stateful modes, there really is no other problem putting statements inside expressions. That's kind of the opposite of the way people usually try to tackle this problem, but if Guido's right that the usual way is inherently bound to fail...
Andrew Barnert
Your old post that Franklin linked explains, given that we have that distinction, and significant indentation for statements, why any attempt to embed statements inside expressions is going to be inherently complex, and therefore likely a bad idea.
But what I was hoping for was a post where you explained why we have that distinction in the first place. If Ruby and CoffeeScript don't need statements, why does Python? I think there's an answer, and it's part of the reason why code in Python tends to be more readable than in those languages (although, as you pointed out, CoffeeScript has other tangential problems), but I was hoping for _your_ answer.
Guido van Rossum
I'm arguing the same thing: anyone who wants to embed statements inside expressions needs to understand why statements and expressions are distinct in the first place.
Your old post that Franklin linked explains, given that we have that distinction, and significant indentation for statements, why any attempt to embed statements inside expressions is going to be inherently complex, and therefore likely a bad idea.
But what I was hoping for was a post where you explained why we have that distinction in the first place. If Ruby and CoffeeScript don't need statements, why does Python? I think there's an answer, and it's part of the reason why code in Python tends to be more readable than in those languages (although, as you pointed out, CoffeeScript has other tangential problems), but I was hoping for _your_ answer.
So it's hard to say to what extent this was a conscious choice and to what extent it was just tradition. But there's nothing necessarily wrong with tradition (up to a point). I think it still makes sense that statements are laid out vertically while expressions are laid out horizontally. Come to think of it, mathematics uses a similar convention -- a formula is laid out (primarily) horizontally, while a sequence of formulas (like a proof or a set of axioms) is laid out vertically.
Yawar Amin
On 2015-01-20 12:15, Franklin? Lee wrote:
> [...]
> http://www.artima.com/weblogs/viewpost.jsp?thread=147358
before starting this thread :-) To be honest, after reading it I came
away thinking that language design/API design/whatever technical design
really _can_ be just solving a puzzle--if you encode unseability as one
of the parameters of the puzzle.
> Yawar might also need this, if he chooses to pursue this path.
> https://wiki.python.org/moin/AlternateLambdaSyntax
all rather unfeasible for the simple reason that Python doesn't allow
statements inside of expressions (nor should it--I believe; it would not
be a good fit for the current syntax). The closest I've seen to one of
those alternate lamba syntaxes being implemented is in the Mochi
language, which compiles down to Python 3 bytecode.
Personally, I've gone in a different direction--Google my name + lambdak
and you will find it :-)
Regards,
Yawar
Nick Coghlan
> So it's hard to say to what extent this was a conscious choice and to what
> extent it was just tradition. But there's nothing necessarily wrong with
> tradition (up to a point). I think it still makes sense that statements are
> laid out vertically while expressions are laid out horizontally. Come to
> think of it, mathematics uses a similar convention -- a formula is laid out
> (primarily) horizontally, while a sequence of formulas (like a proof or a
> set of axioms) is laid out vertically.
mathematical proof" analogy.
Cheers,
Ron Adam
On 01/23/2015 08:56 AM, Nick Coghlan wrote:
> On 21 January 2015 at 11:16, Guido van Rossum<gu...@python.org> wrote:
>> >So it's hard to say to what extent this was a conscious choice and to what
>> >extent it was just tradition. But there's nothing necessarily wrong with
>> >tradition (up to a point). I think it still makes sense that statements are
>> >laid out vertically while expressions are laid out horizontally. Come to
>> >think of it, mathematics uses a similar convention -- a formula is laid out
>> >(primarily) horizontally, while a sequence of formulas (like a proof or a
>> >set of axioms) is laid out vertically.
> I really like that "each statement is like a single step in a
> mathematical proof" analogy.
There are some other distinctions. When you consider these, you can see
how many language designers come to similar solutions.
Expressions evaluate in unique name spaces, while statements generally do
not. Consider "a + b"; it is evaluated in a private method after the values
a and b are passed to it.
Statements are used to mutate the current name space, while expressions
generally do not.
Statements can alter control flow, while expressions generally do not.
Having a clear distinction between expressions and statements makes reading
and understanding code much easier.
I believe Python follows most of these conventions in most places, and when
it doesn't, it's usually for a practical reason that are fairly obvious.
For example, an "or" expression is a bit of both.
Another example of how python chooses a practical alternative is we can
nest expressions instead of using "call" statements on separate lines and a
stack to hold the augments and return values. That is what python does in
the byte code so we don't have to do it in explicit statements.
If you factor out all expressions you get byte code. Or if you factor out
all statements you get something like lisp.
Of course this subject is definitely a very subjective one which relays on
agreeing on the general meaning of the above sentences. OR... YMMV.
Cheers,
Ron
Andrew Barnert
I don't think that's true. Consider "a[b] = c", which is a statement, but it's evaluated in a private method after the values a, b, and c are passed to it. The fact that it's a.__setitem__ rather than a.__add__ doesn't seem particularly important. I think the key is that __setitem__ doesn't have a return value--like (nearly) everything in Python that mutates state--and therefore there's no expression for it. The question is, what does that buy?
> Statements are used to mutate the current name space, while expressions
> generally do not.
This is closer. I think, more generally, statements are used to mutate the important state that the local function is all about mutating. And the thing that gets mutated is almost always the leftmost thing. That definitely helps in scannability.
> Statements can alter control flow, while expressions generally do not.
Sure, but I think this part only really helps if control flow is somehow visible. In Python, it is, because control flow almost always means compound statements, which means indentation, and very little else means indentation.
> Having a clear distinction between expressions and statements makes reading
> and understanding code much easier.
Definitely. That's the part I think is key, but am struggling to explain.
I think Guido offers a great analogy in mathematical proofs, but the question is to find the actual commonality behind the analogy.
After some more thought on this, I think what it comes down to is that (idiomatically-written) Python lets you skim the control flow (because all non-trivial control flow, and very little else, is expressed in terms of indentation) and the state transitions (because each statement generally mutates at most one thing, and it's the leftmost thing), so you can quickly find the part of the code you actually need to read carefully, instead of having to read the whole thing.
I've written this idea up in a bit more detail here:
http://stupidpythonideas.blogspot.com/2015/01/statements-and-expressions.html
> I believe Python follows most of these conventions in most places, and when
> it doesn't, it's usually for a practical reason that are fairly obvious.
>
> For example, an "or" expression is a bit of both.
>
> Another example of how python chooses a practical alternative is we can
> nest expressions instead of using "call" statements on separate lines
> and a
> stack to hold the augments and return values. That is what python does in
> the byte code so we don't have to do it in explicit statements.
>
> If you factor out all expressions you get byte code. Or if you factor out
> all statements you get something like lisp.
I don't think either part of that is true. Bytecode is full of things that are expressions—even your paradigm case of an operator expression is handled by an opcode. And conversely, CoffeeScript (if you avoid break/continue/return statements) factors out all statements, and Ruby comes close to doing so, and yet they're really not more Lisp-like than Python in any meaningful way.
> Of course this subject is definitely a very subjective one which relays on
> agreeing on the general meaning of the above sentences. OR... YMMV.
Ron Adam
On 01/24/2015 12:06 AM, Andrew Barnert wrote:
> On Friday, January 23, 2015 2:06 PM, Ron Adam<ron...@gmail.com>
> wrote:
>
>>>>>
>>>
>>> On 01/23/2015 08:56 AM, Nick Coghlan wrote:
>>>>> I really like that "each statement is like a single step in a
>>>>> mathematical proof" analogy.
>>>
>>> I agree.
>>>
>>>
>>> There are some other distinctions. When you consider these, you can
>>> see how many language designers come to similar solutions.
>>>
>>>
>>> Expressions evaluate in unique name spaces, while statements
>>> generally do not. Consider "a + b"; it is evaluated in a private
>>> method after the values a and b are passed to it.
> I don't think that's true.
Not all the time, which is why I said "generally". :-)
> Consider "a[b] = c", which is a statement,
> but it's evaluated in a private method after the values a, b, and c are
> passed to it. The fact that it's a.__setitem__ rather than a.__add__
> doesn't seem particularly important. I think the key is that __setitem__
> doesn't have a return value--like (nearly) everything in Python that
> mutates state--and therefore there's no expression for it. The question
> is, what does that buy?
Right, and good question, but consider that it's not adding or altering the
name space the 'a' object is in. It's a convenience/exception for using
objects.
If objects were done with closures, then the same modification would be
done with an assignment statement. And then the generalisation would be
more consistent. But at a cost is other ways.
>>> Statements are used to mutate the current name space, while
>>> expressions generally do not.
> This is closer. I think, more generally, statements are used to mutate
> the important state that the local function is all about mutating. And
> the thing that gets mutated is almost always the leftmost thing. That
> definitely helps in scannability.
I'm not quite sure I follow that. I think what you are calling the
important state, I think of as the shared state. A value that will be used
multiple times in the same frame.
Binding a name to a value mutates the names pace, but it does not mutate
the name. It's still the same name.
Again this can be view multiple way if you consider how it actually works.
Some languages use a stack to implement a name. And in a new frame, a new
bound value would get pushed on the name stack. I'm not completely sure
that python doesn't do something like that in some places to speed thing
up. But I don't think so.
>>> Statements can alter control flow, while expressions generally do
>>> not.
> Sure, but I think this part only really helps if control flow is somehow
> visible. In Python, it is, because control flow almost always means
> compound statements, which means indentation, and very little else means
> indentation.
You are referring to the visual aspect, while I'm referring to what a
statement does. Same thing. ;-)
>>> Having a clear distinction between expressions and statements makes
>>> reading and understanding code much easier.
> Definitely. That's the part I think is key, but am struggling to
> explain.
>
> I think Guido offers a great analogy in mathematical proofs, but the
> question is to find the actual commonality behind the analogy.
Form follows function... Just one way to look at it. Sometimes what
something does can come from the shape it has too.
> After some more thought on this, I think what it comes down to is that
> (idiomatically-written) Python lets you skim the control flow (because
> all non-trivial control flow, and very little else, is expressed in
> terms of indentation) and the state transitions (because each statement
> generally mutates at most one thing, and it's the leftmost thing), so
> you can quickly find the part of the code you actually need to read
> carefully, instead of having to read the whole thing.
>
> I've written this idea up in a bit more detail here:
>
> http://stupidpythonideas.blogspot.com/2015/01/statements-and-expressions.html
Very interesting.. and thanks for the mention. ;-)
One of the things that makes a difference is to be able to hold a
simplified model in your mind while you are working on it. The separation
of statements and expressions definitely helps me with that. If I can
easily keep the name space in my mind... or at least the part of it that
correspond with the block of code I'm looking at, It really helps me to
visualise things in a meaningful way and see what effect the statements
will have on the name space.
When statements and expressions don't represent what they do in an obvious
way, then all bets are off. It becomes a mental stumbling block.
Much of this became clear to me when I wrote a simplified script language
that adds statements to a mini scheme like language. I did need to use
braces for blocks. Even though I didn't need to use visual indentation, I
still preferred formatting the programs in a similar style to python. In
this mini language, the separation of statements and expressions is even
more pronounced because all expressions are s-expressions, and all
statements are not expressions.
And going one bit further, I used dictionaries for names spaces, like
python, but used lists for statement blocks. So statement blocks have
order, but name spaces don't.
>>> I believe Python follows most of these conventions in most places,
>>> and when it doesn't, it's usually for a practical reason that are
>>> fairly obvious.
>>>
>>> For example, an "or" expression is a bit of both.
>>>
>>> Another example of how python chooses a practical alternative is we
>>> can nest expressions instead of using "call" statements on separate
>>> lines and a stack to hold the augments and return values. That is
>>> what python does in the byte code so we don't have to do it in
>>> explicit statements.
>>>
>>> If you factor out all expressions you get byte code. Or if you
>>> factor out all statements you get something like lisp.
> I don't think either part of that is true.
It wasn't meant to be taken absolutely literally. Which is why I said...
"something like". There is quite a bit of wiggle room when it comes to how
different people think about things, and what words we use to describe
them. Some times the hardest part of a discussion is getting the same
mental picture in more than one person. :-)
> Bytecode is full of things
> that are expressions—even your paradigm case of an operator expression
> is handled by an opcode.
I see opcodes as being similar to keywords that are used in a statement
form. And I view the stack as being a temporary value space. Of course, I
know certain combinations of several opcodes together may correspond to a
particular python expression, but individually, each byecode along with
it's following few values, are byte code statements to me.
And even if a single bytecode was the equivalent of a python expression, my
point was you can use statements replace statements with expressions.
Byte code is a good example of that. But you still have functions calls..
You just can't nest them in the same way you do with python function calls,
you must push them on the stack and use CALL "statements" to execute it,
and then use another statement to store the value that gets put on the top
of the stack. (or push another function on the stack that will use that
value...)
> And conversely, CoffeeScript (if you avoid
> break/continue/return statements) factors out all statements, and Ruby
> comes close to doing so, and yet they're really not more Lisp-like than
> Python in any meaningful way.
See the next paragraph... ;-)
Andrew Barnert
I fear we're probably getting pretty far off-topic for python-ideas, but I'm going to respond anyway.
> On 01/24/2015 12:06 AM, Andrew Barnert wrote:
>> On Friday, January 23, 2015 2:06 PM, Ron Adam<ron...@gmail.com>
>> wrote:
>>
>>>>
>>>> On 01/23/2015 08:56 AM, Nick Coghlan wrote:
>
>>>>>> I really like that "each statement is like a single step in a
>>>>>> mathematical proof" analogy.
>>>>
>>>> I agree.
>>>>
>>>>
>>>> There are some other distinctions. When you consider these, you can
>>>> see how many language designers come to similar solutions.
>>>>
>>>>
>>>> Expressions evaluate in unique name spaces, while statements
>>>> generally do not. Consider "a + b"; it is evaluated in a private
>>>> method after the values a and b are passed to it.
>
>> I don't think that's true.
>
> Not all the time, which is why I said "generally". :-)
>
>> Consider "a[b] = c", which is a statement,
>> but it's evaluated in a private method after the values a, b, and c are
>> passed to it. The fact that it's a.__setitem__ rather than a.__add__
>> doesn't seem particularly important. I think the key is that __setitem__
>> doesn't have a return value--like (nearly) everything in Python that
>> mutates state--and therefore there's no expression for it. The question
>> is, what does that buy?
>
> Right, and good question, but consider that it's not adding or altering the name space the 'a' object is in. It's a convenience/exception for using objects.
I think that's the point. In theory, objects are just syntactic sugar for closures; in practice, objects are an intuitively useful way to represent mutable state. And the fact that things that mutate an object are statements (whether directly, because they're __setattr__ type calls, or indirectly, because they're expressions that return None and can therefore only be used in expression statements) is important in making Python readable in practice.
> If objects were done with closures, then the same modification would be done with an assignment statement. And then the generalisation would be more consistent. But at a cost is other ways.
>
>>>> Statements are used to mutate the current name space, while
>>>> expressions generally do not.
>> This is closer. I think, more generally, statements are used to mutate
>> the important state that the local function is all about mutating. And
>> the thing that gets mutated is almost always the leftmost thing. That
>> definitely helps in scannability.
>
> I'm not quite sure I follow that. I think what you are calling the important state, I think of as the shared state. A value that will be used multiple times in the same frame.
By "important state", I just mean whatever state the reader is likely to care about. And such state is almost always modified by a statement with some readable identification on the left side--whether it's a global/closure/local assignment, an attribute assignment, an element assignment, and augmented version of any of the above, a method call that doesn't return self (and will therefore only be used in an expression statement). So each statement means (at most) one state transition. And it's almost always the leftmost thing that's affected. And I think that aids readability.
> Binding a name to a value mutates the names pace, but it does not mutate the name. It's still the same name.
>
> Again this can be view multiple way if you consider how it actually works. Some languages use a stack to implement a name. And in a new frame, a new bound value would get pushed on the name stack. I'm not completely sure that python doesn't do something like that in some places to speed thing up. But I don't think so.
Sure, pure non-mutating languages can use a stack to implement bindings. But in a mutating language, that doesn't work if you have closures. CTM explains what you get out of mutable state (and what it costs) very nicely. Of course in practice any Python implementation has to be able to detect whether a given scope might have closures referring to it, so it _could_ switch to rebinding using a bindinmg stack, but in practice any Python implementation is likely to convert local accesses to offsets as CPython does, which is a much better optimization (at least for a language where rebinding is idiomatically common) that precludes that option.
>>>> Statements can alter control flow, while expressions generally do
>>>> not.
>> Sure, but I think this part only really helps if control flow is somehow
>> visible. In Python, it is, because control flow almost always means
>> compound statements, which means indentation, and very little else means
>> indentation.
>
> You are referring to the visual aspect, while I'm referring to what a statement does. Same thing. ;-)
Well, I'm highlighting the visual aspect because I think that's important to Python's readability, and to why statements contribute to that readability. If statements and expressions had similar indentation rules (as in CoffeeScript), I don't think Python would get the same benefit from having statements.
>>>> Having a clear distinction between expressions and statements makes
>>>> reading and understanding code much easier.
>> Definitely. That's the part I think is key, but am struggling to
>> explain.
>>
>> I think Guido offers a great analogy in mathematical proofs, but the
>> question is to find the actual commonality behind the analogy.
>
> Form follows function... Just one way to look at it. Sometimes what something does can come from the shape it has too.
Yes. But sometimes the shape is limiting, rather than expanding--and yet that limitation itself can be used to add meaning. (See Guido's point about code that fits in a window/screen/page.)
So function partly follows form. And mathematical proofs are a great example. The fact that there are a limited number of ways you're allowed to get from the previous statements to the next one makes each step more readable.
>> After some more thought on this, I think what it comes down to is that
>> (idiomatically-written) Python lets you skim the control flow (because
>> all non-trivial control flow, and very little else, is expressed in
>> terms of indentation) and the state transitions (because each statement
>> generally mutates at most one thing, and it's the leftmost thing), so
>> you can quickly find the part of the code you actually need to read
>> carefully, instead of having to read the whole thing.
>>
>> I've written this idea up in a bit more detail here:
>>
>> http://stupidpythonideas.blogspot.com/2015/01/statements-and-expressions.html
>
> Very interesting.. and thanks for the mention. ;-)
Sure; as I said, your last paragraph (well, the last one I quoted) puts the whole thing I'm trying to answer much more clearly than I've been able to.
> One of the things that makes a difference is to be able to hold a simplified model in your mind while you are working on it.
Yes! That's something else that was on the tip of my tongue that I couldn't explain clearly. Being able to hold enough of the syntax in your head to parse code subconsciously (which CoffeeScript lacks, as Guido pointed out) is part of it, but you're right, the big deal is being able to hold the entire model in your head.
And at a different level, that's what makes scanability of flow control and state changes so important, which I couldn't explain before. That's the model of an imperative-style function that you need to be able to internalize to understand the function holistically.
Thanks.
> The separation of statements and expressions definitely helps me with that. If I can easily keep the name space in my mind... or at least the part of it that correspond with the block of code I'm looking at, It really helps me to visualise things in a meaningful way and see what effect the statements will have on the name space.
Exactly. Which is why the "one obvious mutation per line (usually)" property is so important.
But I think describing it purely in terms of the local namespace hides the fact that it applies just as well to OO-style code (where most mutation is to the namespace of self or one of the other parameters) as to traditional structured imperative code.
> When statements and expressions don't represent what they do in an obvious way, then all bets are off. It becomes a mental stumbling block.
> Much of this became clear to me when I wrote a simplified script language that adds statements to a mini scheme like language. I did need to use braces for blocks. Even though I didn't need to use visual indentation, I still preferred formatting the programs in a similar style to python. In this mini language, the separation of statements and expressions is even more pronounced because all expressions are s-expressions, and all statements are not expressions.
>
> And going one bit further, I used dictionaries for names spaces, like python, but used lists for statement blocks. So statement blocks have order, but name spaces don't.
>
>
>>>> I believe Python follows most of these conventions in most places,
>>>> and when it doesn't, it's usually for a practical reason that are
>>>> fairly obvious.
>>>>
>>>> For example, an "or" expression is a bit of both.
>>>>
>>>> Another example of how python chooses a practical alternative is we
>>>> can nest expressions instead of using "call" statements on separate
>>>> lines and a stack to hold the augments and return values. That is
>>>> what python does in the byte code so we don't have to do it in
>>>> explicit statements.
>>>>
>>>> If you factor out all expressions you get byte code. Or if you
>>>> factor out all statements you get something like lisp.
>> I don't think either part of that is true.
>
> It wasn't meant to be taken absolutely literally. Which is why I said... "something like".
OK, but I don't think it's figuratively true in any useful sense either. Languages like Ruby prove that removing statements doesn't have to mean something like Lisp. We aren't stuck with the models of the 60s. So the question of what you give up by factoring out all statements turns out to be more complicated (and more interesting) than it was in those models.
> There is quite a bit of wiggle room when it comes to how different people think about things, and what words we use to describe them. Some times the hardest part of a discussion is getting the same mental picture in more than one person. :-)
>
>> Bytecode is full of things
>> that are expressions—even your paradigm case of an operator expression
>> is handled by an opcode.
>
> I see opcodes as being similar to keywords that are used in a statement form. And I view the stack as being a temporary value space. Of course, I know certain combinations of several opcodes together may correspond to a particular python expression, but individually, each byecode along with it's following few values, are byte code statements to me.
>
> And even if a single bytecode was the equivalent of a python expression, my point was you can use statements replace statements with expressions.
I don't know what you meant here. Maybe that a stack machine language has a fixed, non-extensible set of expressions, but its set of statements can be effectively arbitrarily extended with jsr/ret to other bytecode? If so, I'll buy that, but I'm not sure how it's relevant. I don't know of any examples of readable expression-free code that compare to such examples of readable statement-free code as, say, anything written in OCaml, or any Ruby code that sticks to single-return style. Eliminating expressions is clearly a non-starter for a readable language; eliminating statements is actually arguable--and the whole point is to find the arguments against doing so.
> Byte code is a good example of that. But you still have functions calls.. You just can't nest them in the same way you do with python function calls, you must push them on the stack and use CALL "statements" to execute it, and then use another statement to store the value that gets put on the top of the stack. (or push another function on the stack that will use that value...)
>
>
>> And conversely, CoffeeScript (if you avoid
>> break/continue/return statements) factors out all statements, and Ruby
>> comes close to doing so, and yet they're really not more Lisp-like than
>> Python in any meaningful way.
>
> See the next paragraph... ;-)
>
>>>> Of course this subject is definitely a very subjective one which
>>>> relays on agreeing on the general meaning of the above sentences.
>>>> OR... YMMV.
Sure, but I'm not sure I understand the meaning you're going for.
And, more importantly, I think there is an objective sense in which Python uses statements to gain subjective readability, and that objective sense is something we can question and try to answer.
Guido van Rossum
Nick Coghlan
On 25 Jan 2015 03:34, "Guido van Rossum" <gu...@python.org> wrote:
>
> Can you guys get a room?
More explicitly - I think we've established we *don't actually know* why the statement/expression distinction improves readability, just that decades of experience suggests it does.
Further speculation here is unlikely to bring further clarity, so anyone interested in seeing it pursued to the point of reaching a more definitive conclusion would be well advised to get in touch with folks that actually research programming language readability (e.g. the authors of this paper from a few years ago: http://neverworkintheory.org/2011/10/24/an-empirical-comparison-of-the-accuracy-rates-of-novices-using-the-quorum-perl-and-randomo-programming-languages.html and their more recent follow-up http://neverworkintheory.org/2014/01/29/stefik-siebert-syntax.html)
Cheers,
Nick.
Yawar Amin
On 2015-01-24 20:54, Nick Coghlan wrote:
> On 25 Jan 2015 03:34, "Guido van Rossum" <gu...@python.org> wrote:
> >
> > Can you guys get a room?
> that actually research programming language readability (e.g. the
> authors of this paper from a few years ago:
> http://neverworkintheory.org/2011/10/24/an-empirical-comparison-of-the-accuracy-rates-of-novices-using-the-quorum-perl-and-randomo-programming-languages.html
> and their more recent follow-up
> http://neverworkintheory.org/2014/01/29/stefik-siebert-syntax.html)
> >> > On 01/24/2015 12:06 AM, Andrew Barnert wrote:
> >> >>
> >> >> http://stupidpythonideas.blogspot.com/2015/01/statements-and-expressions.html
Regards,
Yawar