[Python-ideas] Inline assignments using "given" clauses

[Nick Coghlan]

(Note: Guido's already told me off-list that he doesn't like the way this spelling reads, but I wanted to share it anyway since it addresses one of the recurring requests in the PEP 572 discussions for a more targeted proposal that focused specifically on the use cases that folks had agreed were reasonable potential use cases for inline assignment expressions.

I'll also note that another potential concern with this specific proposal is that even though "given" wasn't used as a term in any easily discovered Python APIs back when I first wrote PEP 3150, it's now part of the Hypothesis testing API, so adopting it as a keyword now would be markedly more disruptive than it might have been historically)

Recapping the use cases where the inline assignment capability received the most agreement regarding being potentially more readable than the status quo:

1. Making an "exactly one branch is executed" construct clearer than is the case for nested if statements:

    if m := pattern.search(data):
        ...

    elif m := other_pattern.search(data):
        ...

    else:
        ...

2. Replacing a loop-and-a-half construct:

    while m := pattern.search(remaining_data):
        ...

3. Sharing values between filtering clauses and result expressions in comprehensions:

    result = [(x, y, x/y) for x in data if (y := f(x))]

The essence of the given clause concept would be to modify *these specific cases* (at least initially) to allow the condition expression to be followed by an inline assignment, of the form "given TARGET = EXPR". (Note: being able to implement such a syntactic constraint is a general consequence of using a ternary notation rather than a binary one, since it allows the construct to start with an arbitrary expression, without requiring that expression to be both the result of the operation *and* the value bound to a name - it isn't unique to the "given" keyword specifically)

While the leading keyword would allow TARGET to be an arbitrary assignment target without much chance for confusion, it could also be restricted to simple names instead (as has been done for PEP 572.

With that spelling, the three examples above would become:

    # Exactly one branch is executed here

    if m given m = pattern.search(data):
        ...

    elif m given m = other_pattern.search(data)):
        ...

    else:
        ...

    # This name is rebound on each trip around the loop

    while m given m = pattern.search(remaining_data):
        ...

    # "f(x)" is only evaluated once on each iteration

    result = [(x, y, x/y) for x in data if y given y = f(x)]

Constraining the syntax that way (at least initially) would avoid poking into any dark corners of Python's current scoping and expression execution ordering semantics, while still leaving the door open to later making "result given NAME = expr" a general purpose ternary operator that returns the LHS, while binding the RHS to the given name as a side effect.

Using a new keyword (rather than a symbol) would make the new construct easier to identify and search for, but also comes with all the downsides of introducing a new keyword. (Hence the not-entirely-uncommon suggestion of using "with" for a purpose along these lines, which runs into a different set of problems related to trying to use "with" for two distinct and entirely unrelated purposes).

Cheers,

Nick.

--

Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia

[Guido van Rossum]

Thanks! Perhaps the most useful bit of this post is the clear list of use cases (a useful summary of the motivational part of PEP 572).

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--

--Guido van Rossum (python.org/~guido)

[Serhiy Storchaka]

04.05.18 15:06, Nick Coghlan пише:

> Recapping the use cases where the inline assignment capability received
> the most agreement regarding being potentially more readable than the
> status quo:

Sorry, but these examples don't look as good examples for inline
assignments to me. I think that all these cases can be written better
without using the inline assignment.

> 1. Making an "exactly one branch is executed" construct clearer than is
> the case for nested if statements:
>
>     if m := pattern.search(data):
>         ...
>     elif m := other_pattern.search(data):
>         ...
>     else:
>         ...

This case can be better handled by combining patterns in a single
regular expression.

pattern = re.compile('(?P<foo>pattern1)|(?P<bar>pattern2)|...')
m = pattern.search(data)
if not m: # this can be omitted if the pattern is always found
...
elif m.group('foo'):
...
elif m.group('bar'):
...

See for example gettext.py where this pattern is used.

> 2. Replacing a loop-and-a-half construct:
>
>     while m := pattern.search(remaining_data):
>         ...

This case can be better handled by re.finditer().

for m in pattern.finditer(remaining_data):
...

In more complex cases it is handy to write a simple generator function
and iterate its result.

The large number of similar cases are covered by a two-argument iter().

> 3. Sharing values between filtering clauses and result expressions in
> comprehensions:
>
>     result = [(x, y, x/y) for x in data if (y := f(x))]

There are a lot of ways of writing this. PEP 572 mentions them.
Different ways are used in real code depending on preferences of the
author. Actually the number of these cases is pretty low in comparison
with the total number of comprehensions.

It is possible to express an assignment in comprehensions with the "for
var in [value]" idiom, and this idiom is more powerful than PEP 572 in
this case because it allows to perform an assignment before the first
'for'. But really complex comprehensions could be better written as
'for' statements with explicit adding to the collection or yielding.

[Tim Peters]

[Nick Coghlan <ncog...@gmail.com>]
> ...

> Using a new keyword (rather than a symbol) would make the new construct
> easier to identify and search for, but also comes with all the downsides of
> introducing a new keyword.

That deserves more thought. I started my paying career working on a
Fortran compiler, a language which, by design, had no reserved words
(although plenty of keywords). The language itself (and
vendor-specific extensions) never had to suffer "but adding a new
keyword could break old code!" consequences.

In practice that worked out very well, Yes, you _could_ write
hard-to-read code using language keywords as, e.g., identifier names
too, but, no, absolutely nobody did that outside of "stupid Fortran
tricks" posts on Usenet ;-) It had the _intended_ effect in practice:
no breakage of old code just because the language grew new
constructs.

It's no longer the case that Python avoided that entirely, since
"async def", "async for", and "async with" statements were added
_without_ making "async" a new reserved word. It may require pain in
the parser, but it's often doable anyway. At this stage in Python's
life, adding new _reserved_ words "should be" an extremely high bar -
but adding new non-reserved keywords (like "async") should be a much
lower bar.

That said, I expect it's easier in general to add a non-reserved
keyword introducing a statement (like "async") than one buried inside
expressions ("given").

[Guido van Rossum]

On Fri, May 4, 2018 at 11:11 AM, Tim Peters <tim.p...@gmail.com> wrote:

[Nick Coghlan <ncog...@gmail.com>]
> ...
> Using a new keyword (rather than a symbol) would make the new construct
> easier to identify and search for, but also comes with all the downsides of
> introducing a new keyword.

That deserves more thought.  I started my paying career working on a
Fortran compiler, a language which, by design, had no reserved words
(although plenty of keywords).  The language itself (and
vendor-specific extensions) never had to suffer "but adding a new
keyword could break old code!" consequences.

In practice that worked out very well,  Yes, you _could_ write
hard-to-read code using language keywords as, e.g., identifier names
too, but, no, absolutely nobody did that outside of "stupid Fortran
tricks" posts on Usenet ;-)  It had the _intended_ effect in practice:
 no breakage of old code just because the language grew new
constructs.

It's no longer the case that Python avoided that entirely, since
"async def", "async for", and "async with" statements were added
_without_ making "async" a new reserved word.  It may require pain in
the parser, but it's often doable anyway.  At this stage in Python's
life, adding new _reserved_ words "should be" an extremely high bar -
but adding new non-reserved keywords (like "async") should be a much
lower bar.

Do note that this was a temporary solution. In 3.5 we introduced this hack. In 3.6, other uses of `async` and `await` became deprecated (though you'd have to use `python -Wall` to get a warning). In 3.7, it's a syntax error.

 

That said, I expect it's easier in general to add a non-reserved
keyword introducing a statement (like "async") than one buried inside
expressions ("given").

I'd also say that the difficulty of Googling for the meaning of ":=" shouldn't be exaggerated. Currently you can search for "python operators" and get tons of sites that list all operators.

I also note that Google seems to be getting smarter about non-alphabetic searches -- I just searched for "python << operator" and the first hit was https://wiki.python.org/moin/BitwiseOperators

[Matt Arcidy]

Without adding hits to the search algorithm, this will remain the case.  Google must have clicks to rank up.  Right now there is no page, nothing on a high "Google juice" page like python.org, no one searching for it, and no mass of people clicking on it.  no SO questions, etc.

there is a transient response for all change.  uniqueness and length of search term is just a faster one.  All python syntax is findable eventually due to popularity.  plus a better search is "why would I use...in python" or similar.

= python also doesn't bring up anything interesting that wouldn't be had because of just "python".  The details are too mundane and/or technical and everyone knows already.

that being said, if := had been (theoretically) included from the beginning, would people continue to have issues with it?  unlikely, but I can't know.  familiarity will cure many of these issues of readability or symbolic disagreement no matter what is chosen (well, to a point).  it's unfortunate that changes have to be made up front with so little information like that, so I'm not advocating anything based on this, just pointing it out.

I do think post hoc assignment will cause a cognitive load, like trying to figure out which variable is the iterator, and having to keep two contexts till the end of a comp with one given statement.  

[f(x) + a for all a in blah given x=1]  

not worse than a double nested comp though.

I also note that Google seems to be getting smarter about non-alphabetic searches -- I just searched for "python << operator" and the first hit was https://wiki.python.org/moin/BitwiseOperators

--

--Guido van Rossum (python.org/~guido)

[Chris Angelico]

On Sat, May 5, 2018 at 5:27 AM, Matt Arcidy <mar...@gmail.com> wrote:
>> I'd also say that the difficulty of Googling for the meaning of ":="
>> shouldn't be exaggerated. Currently you can search for "python operators"
>> and get tons of sites that list all operators.
>
>
> Without adding hits to the search algorithm, this will remain the case.
> Google must have clicks to rank up. Right now there is no page, nothing on
> a high "Google juice" page like python.org, no one searching for it, and no
> mass of people clicking on it. no SO questions, etc.

Did you try? I searched for 'python :=' and for 'python colon equals'
and got this hit each time:

https://stackoverflow.com/questions/26000198/what-does-colon-equal-in-python-mean

Which, incidentally, now has a response to it citing PEP 572.

Good ol' Stack Overflow.

ChrisA

[Tim Peters]

[Tim]
>> ...

>> It's no longer the case that Python avoided that entirely, since
>> "async def", "async for", and "async with" statements were added
>> _without_ making "async" a new reserved word. It may require pain in
>> the parser, but it's often doable anyway. At this stage in Python's
>> life, adding new _reserved_ words "should be" an extremely high bar -
>> but adding new non-reserved keywords (like "async") should be a much
>> lower bar.

[Guido]

> Do note that this was a temporary solution. In 3.5 we introduced this hack.
> In 3.6, other uses of `async` and `await` became deprecated (though you'd
> have to use `python -Wall` to get a warning). In 3.7, it's a syntax error.

See my "that deserves more thought" at the start, but wrt future cases
then ;-) In 3.5 and 3.6, "everything just works" for everyone. In
3.7 the implementation gets churned again, to go out of its way to
break the handful of code using "async" as an identifier. It's
obvious who that hurts, but who does that really benefit?

My experience with Fortran convinces me nobody would _actually_ be
confused even if they wrote code like:

async def frobnicate(async=True):
if async:
async with ...

But nobody would actually do that. Then again, "but people _could_ do
that!" barely registers with me because the nobody-actually-does-it
theoretical possibilities were so much worse in Fortran, so I tend to
tune that kind of argument out reflexively. For example, whitespace
was also irrelevant in Fortran, and these two statements mean
radically different things:

D O1 0I=1 00,30 0
D O1 0I=1 00.30 0

The first is like:

for I in range(100, 301):
# the block ends at the next statement with label 10

The seconds is like:

DO10I = 100.300

All actual Fortran code spells them like this instead:

DO 10 I = 100, 300
DO10I = 100.300

The differing intents are obvious at a glance then - although, yup, to
the compiler the difference is solely due to that one uses a comma
where the other a period. I'm not suggesting Python go anywhere near
_that_ far ;-) Just as far as considering that there's no actual harm
in Fortran allowing "DO" to be a variable name too. Nobody is even
tempted to think that "DO" might mean "DO loop" in, e.g.,

DO = 4
X = FUNC(DO)
X = DO(Y)
IF (DO.OR.DONTDO) GOTO 10

etc. People generally _don't_ use Fortran keywords as identifiers
despite that they can, but it's a real boon for the relatively rare
older code that failed to anticipate keywords added after it was
written.

> ...

> I'd also say that the difficulty of Googling for the meaning of ":="
> shouldn't be exaggerated. Currently you can search for "python operators"
> and get tons of sites that list all operators.

I've noted before that people don't seem to have trouble finding the
meaning of Python's "is", "and", and "or" either. But Googling for
"is" (etc) on its own isn't the way to do it ;-)

> I also note that Google seems to be getting smarter about non-alphabetic
> searches -- I just searched for "python << operator" and the first hit was
> https://wiki.python.org/moin/BitwiseOperators

Ya - most arguments are crap ;-)

[Nathaniel Smith]

On Fri, May 4, 2018 at 1:53 PM, Tim Peters <tim.p...@gmail.com> wrote:
> [Tim]
>>> ...
>>> It's no longer the case that Python avoided that entirely, since
>>> "async def", "async for", and "async with" statements were added
>>> _without_ making "async" a new reserved word. It may require pain in
>>> the parser, but it's often doable anyway. At this stage in Python's
>>> life, adding new _reserved_ words "should be" an extremely high bar -
>>> but adding new non-reserved keywords (like "async") should be a much
>>> lower bar.
>
> [Guido]
>> Do note that this was a temporary solution. In 3.5 we introduced this hack.
>> In 3.6, other uses of `async` and `await` became deprecated (though you'd
>> have to use `python -Wall` to get a warning). In 3.7, it's a syntax error.
>
> See my "that deserves more thought" at the start, but wrt future cases
> then ;-) In 3.5 and 3.6, "everything just works" for everyone. In
> 3.7 the implementation gets churned again, to go out of its way to
> break the handful of code using "async" as an identifier. It's
> obvious who that hurts, but who does that really benefit?
>
> My experience with Fortran convinces me nobody would _actually_ be
> confused even if they wrote code like:
>
> async def frobnicate(async=True):
> if async:
> async with ...

IIUC, Javascript has also gone all-in on contextual keywords. The
realities of browser deployment mean they simply cannot have flag days
or break old code, ever, meaning that contextual keywords are really
the only kind they can add at all. So their async/await uses the same
kind of trick that Python 3.5 did, and I believe they plan to keep it
that way forever.

FWIW.

-n

--
Nathaniel J. Smith -- https://vorpus.org

[Guido van Rossum]

I hope Python never has to go there. It's a tooling nightmare.

[Alexander Belopolsky]

On Fri, May 4, 2018 at 8:06 AM, Nick Coghlan <ncog...@gmail.com> wrote:
> ...

> With that spelling, the three examples above would become:
>
> # Exactly one branch is executed here
> if m given m = pattern.search(data):
> ...
> elif m given m = other_pattern.search(data)):
> ...
> else:
> ...
>
> # This name is rebound on each trip around the loop
> while m given m = pattern.search(remaining_data):
> ...
>
> # "f(x)" is only evaluated once on each iteration
> result = [(x, y, x/y) for x in data if y given y = f(x)]

I think this is a step in the right direction. I stayed away from the
PEP 572 discussions because while intuitively it felt wrong, I could
not formulate what exactly was wrong with the assignment expressions
proposals. This proposal has finally made me realize why I did not
like PEP 572. The strong expression vs. statement dichotomy is one of
the key features that set Python apart from many other languages and
it makes Python programs much easier to understand. Right from the
title, "Assignment Expressions", PEP 572 was set to destroy the very
feature that in my view is responsible for much of Python's success.

Unlike PEP 572, Nick's proposal does not feel like changing the syntax
of Python expressions, instead it feels like an extension to the if-,
while- and for-statements syntax. (While comprehensions are
expressions, for the purposes of this proposal I am willing to view
them as for-statements in disguise.)

[Tim Peters]

[Nick Coghlan <ncog...@gmail.com>]
> ...

> The essence of the given clause concept would be to modify *these specific
> cases* (at least initially) to allow the condition expression to be followed
> by an inline assignment, of the form "given TARGET = EXPR".

I'm not clear on what "these specific cases" are, specifically ;-)
Conditions in "if", "elif", and "while" statement expressions?
Restricted to one "given" clause, or can they chain? In a listcomp,
is it one "given" clause per "if", or after at most one "if"? Or is
an "if" even needed at all in a listcomp? For example,

[(f(x)**2, f(x)**3) for x in xs]

has no conditions, and

[(fx := f(x))**2, fx**3) for x in xs]

is one reasonable use for binding expressions.

[(fx**2, fx**3) for x in xs given fx = f(x)]

reads better, although it's initially surprising (to my eyes) to find
fx defined "at the end". But no more surprising than the current:

[(fx**2, fx**3) for x in xs for fx in [f(x)]]

trick.


>....

> While the leading keyword would allow TARGET to be an arbitrary assignment
> target without much chance for confusion, it could also be restricted to
> simple names instead (as has been done for PEP 572.

The problem with complex targets in general assignment expressions is
that, despite trying, I found no plausible use case for (at least)
unpacking syntax. As in, e.g.,

x, y = func_returning_twople()
if x**2 + y**2 > 9: # i.e., distance > 3, but save expensive sqrt

The names can be _unpacked_ in a general assignment expression, but
there appears to be no sane way then to _use_ the names in the test.
This may be as good as it gets:

if [(x, y := func_returning_twople()). x**2 + y**2 > 9][-1]:

That reminds me of the hideous

(condition and [T] or [F])[0]

idiom I "invented" long ago to get the effect (in all cases) of the current

T if condition else F

That was intended to be goofy fun at the time, but I was appalled to
see people later use it ;-)

It''s certain sanest as

if x**2 + y**2 > 9 given x, y = func_returning_twople():

"given" really shines there!

> With that spelling, the three examples above would become:
>
> # Exactly one branch is executed here
> if m given m = pattern.search(data):
> ...
> elif m given m = other_pattern.search(data)):
> ...
> else:

Which is OK. The one-letter variable name obscures that it doesn't
actually reduce _redundancy_, though. That is, in the current

match = pattern.search(data)
if match:

it's obviously less redundant typing as:

if match := pattern.search(data):

In

if match given match = pattern.search(data):

the annoying visual redundancy (& typing) persists.

> # This name is rebound on each trip around the loop
> while m given m = pattern.search(remaining_data):

Also fine, but also doesn't reduce redundancy.

> # "f(x)" is only evaluated once on each iteration
> result = [(x, y, x/y) for x in data if y given y = f(x)]

As above, the potential usefulness of "given" in a listcomp doesn't
really depend on having a conditional. Or on having a listcomp
either, for that matter ;-)

r2, r3 = fx**2, fx**3 given fx = f(x)


One more, a lovely (to my eyes) binding expression simplification
requiring two bindings in an `if` test, taken from real-life code I
happened to write during the PEP discussion:

diff = x - x_base
if diff:
g = gcd(diff, n)
if g > 1:
return g

collapsed to the crisp & clear:

if (diff := x - x_base) and (g := gcd(diff, n)) > 1:
return g

If only one trailing "given" clause can be given per `if` test
expression, presumably I couldn't do that without trickery. If it's
more general,

if (diff given diff = x _ xbase) and g > 1 given g = gcd(diff, n):

reads worse to my eyes (perhaps because of the "visual redundancy"
thing again), while

if diff and g > 1 given diff = x - x_base given g = gcd(diff, n):

has my eyes darting all over the place, and wondering which of the
trailing `given` clauses executes first.

> ...

[Nathaniel Smith]

This is what makes me uncomfortable too. As Dijkstra once wrote:

"our intellectual powers are rather geared to master static relations
and ... our powers to visualize processes evolving in time are
relatively poorly developed. For that reason we should do (as wise
programmers aware of our limitations) our utmost to shorten the
conceptual gap between the static program and the dynamic process, to
make the correspondence between the program (spread out in text space)
and the process (spread out in time) as trivial as possible." [1]

Normally, Python code strongly maps *time* onto *vertical position*:
one side-effect per line. Of course there is some specific
order-of-operations for everything inside an individual line that the
interpreter has to keep track of, but I basically never have to care
about that myself. But by definition, := involves embedding
side-effects within expressions, so suddenly I do have to care after
all. Except... for the three cases Nick wrote above, where the
side-effect occurs at the very end of the evaluation. And these also
seem to be the three cases that have the most compelling use cases
anyway. So restricting to just those three cases makes it much more
palatable to me.

(I won't comment on Nick's actual proposal, which is a bit more
complicated than those examples, since it allows things like 'if
m.group(1) given m = ...'.)

(And on another note, I also wonder if all this pent-up desire to
enrich the syntax of comprehensions means that we should add some kind
of multi-line version of comprehensions, that doesn't require the
awkwardness of explicitly accumulating a list or creating a nested
function to yield out of. Not sure what that would look like, but
people sure seem to want it.)

-n

[1] This is from "Go to statement considered harmful". Then a few
lines later he uses a sequence of assignment statements as an example,
and says that the wonderful thing about this example is that there's a
1-1 correspondence between lines and distinguishable program states,
which is also uncannily apropos.

--
Nathaniel J. Smith -- https://vorpus.org

[Greg Ewing]

Tim Peters wrote:
> "async def", "async for", and "async with" statements were added
> _without_ making "async" a new reserved word. It may require pain in
> the parser, but it's often doable anyway.

There would be less pain if the parser generator could
deal with non-reserved keywords more directly.

--
Greg

[Robert Vanden Eynde]

I agree it would be useful to have new keywords without being reserved, and we could even go with mechanism like infix operators created by user.

It would allow things like [given for x in range(5) given given = x+1] or even [given for given in range(given) given given = given + 1] haha, but as other pointed out, it would be called Bad practice ^^

By the way, I still prefer "where" to "given".

[Juancarlo Añez]

I think that "expr as y"  was discarded too quickly.

It would be a syntax completely familiar to Python programmers, and the new semantics would be obvious. That choice would also be in line with the Zen of Python.

The special cases that may arise over "except"  and "with"  can be worked out and documented.

Cheers,

--

Juancarlo Añez

[Mikhail V]

On Fri, May 4, 2018 at 3:06 PM, Nick Coghlan <ncog...@gmail.com> wrote:

>
> With that spelling, the three examples above would become:
>
> # Exactly one branch is executed here
> if m given m = pattern.search(data):
> ...
> elif m given m = other_pattern.search(data)):
> ...
> else:
> ...
>
> # This name is rebound on each trip around the loop
> while m given m = pattern.search(remaining_data):
> ...
>
> # "f(x)" is only evaluated once on each iteration
> result = [(x, y, x/y) for x in data if y given y = f(x)]
>

Well, I think it looks good. Nice job Nick!
I like that it leaves the "=" alone finally (but it's maybe the
reason why I am biased towards this syntax :)
It has clear look so as not to confuse with conditionals and
the variable is emphasized so it's well understood what it means.

I wish I could like the whole idea of inline assignments more now -
but well, it is just what it is. Something that can help on rare occasion.
But as said, your variant feels right to me. Unlike with most other
proposed constructs here my first impression - this is Python.

If the new keyword is too much, there are some options too choose from.
(though I've no idea whether this is plausible due to parsing ambiguities)

if m given m = pattern.search(data): vs:
if m with m = pattern.search(data):
if m def m = pattern.search(data):
if m as m = pattern.search(data):

Yes, all these reads strange, but IMO understandable.

BTW, using your keyword, in comprehensions vs the original idea
of Chris then merely I suppose:

result = [(x, y, x/y) given y = f(x) for x in data if y ] vs
result = [(x, y, x/y) with y = f(x) for x in data if y ]
result = [(x, y, x/y) def y = f(x) for x in data if y ]
result = [(x, y, x/y) as y = f(x) for x in data if y ]
vs
result = [(x, y, x/y) for x in data if y given y = f(x)] vs
result = [(x, y, x/y) for x in data if y with y = f(x)]
result = [(x, y, x/y) for x in data if y def y = f(x)]
result = [(x, y, x/y) for x in data if y as y = f(x)]

I can't say for sure what is better but the former order seems to
be slightly more appealing for some reson.


[Tim]

> it's obviously less redundant typing as:
> if match := pattern.search(data):
> In
> if match given match = pattern.search(data):

I disagree, I think it is not redundant but it's just how it is -
namely it does not introduce 'implicitness' in the first place,
and that is what I like about Nick's variant. But of course it is less compact.



PS:

recently I have discovered interesting fact: it seems one
can already use 'inline assignment' with current syntax. E.g.:

if exec("m = input()") or m:
print (m)

It seem to work as inline assignment correctly.
Yes it is just coincidence, BUT: this has the wanted effect!
- hides one line in "if" and "while"
- leaves the assignment statement (or is it expression now? ;)
- has explicit appearance

So we have it already?


Mikhail

[Nick Coghlan]

On 6 May 2018 at 00:33, Mikhail V <mikha...@gmail.com> wrote:

recently I have discovered interesting fact: it seems one
can already use 'inline assignment' with current syntax. E.g.:

if exec("m = input()") or m:
    print (m)

It seem to work as inline assignment correctly.
Yes it is just coincidence, BUT: this has the wanted effect!
- hides one line in "if" and "while"
- leaves the assignment statement (or is it expression now? ;)
- has explicit appearance

So we have it already?

Not really, since:

1. exec with a plain string has a very high runtime cost (the compiler is pretty slow, since we assume the results will be cached)

2. exec'ed strings are generally opaque to static analysis tools

3. writing back to the local namespace via exec doesn't work consistently at function scope (and assuming PEP 558 is eventually accepted, will some day reliably *never* work)

Cheers,

Nick.

[Serhiy Storchaka]

05.05.18 18:26, Nick Coghlan пише:

> 3. writing back to the local namespace via exec doesn't work
> consistently at function scope (and assuming PEP 558 is eventually
> accepted, will some day reliably *never* work)

At end you can convert the source of the whole script to a string
literal and write the whole script in one line. The problem solved!

[Nick Coghlan]

On 5 May 2018 at 13:36, Tim Peters <tim.p...@gmail.com> wrote:

[Nick Coghlan <ncog...@gmail.com>]
> ...
> The essence of the given clause concept would be to modify *these specific
> cases* (at least initially) to allow the condition expression to be followed
> by an inline assignment, of the form "given TARGET = EXPR".

I'm not clear on what "these specific cases" are, specifically ;-)
Conditions in "if", "elif", and "while" statement expressions?

Exactly the 3 cases presented (if/elif/while conditions). The usage in comprehensions would mirror the usage in if statements, and avoid allowing name bindings in arbitrary locations due to the implicity nested scopes used by comprehensions.

Conditional expressions would be initially omitted since including them would allow arbitrary name bindings in arbitrary locations via the quirky "x if x given x = expr else x" spelling, and because "else" isn't as distinctive an ending token as "given ... :", "given ... )", "given ... ]", or "given ... }".

 

Restricted to one "given" clause, or can they chain?  In a listcomp,
is it one "given" clause per "if", or after at most one "if"?  Or is
an "if" even needed at all in a listcomp?  For example,

    [(f(x)**2, f(x)**3) for x in xs]

has no conditions, and

    [(fx := f(x))**2, fx**3) for x in xs]

is one reasonable use for binding expressions.

    [(fx**2, fx**3) for x in xs given fx = f(x)]

reads better, although it's initially surprising (to my eyes) to find
fx defined "at the end".  But no more surprising than the current:

    [(fx**2, fx**3) for x in xs for fx in [f(x)]]

trick.

There were a couple key reasons I left the "for x in y" case out of the initial proposal:

1. The "for x in y" header is already quite busy, especially when tuple unpacking is used in the assignment target

2. Putting the "given" clause at the end would make it ambiguous as to whether it's executed once when setting up the iterator, or on every iteration

3. You can stick in an explicit "if True" if you don't need the given variable in the filter condition

    [(fx**2, fx**3) for x in xs if True given fx = f(x)]

And then once you've had an entire release where the filter condition was mandatory for the comprehension form, allowing the "if True" in "[(fx**2, fx**3) for x in xs given fx = f(x)]" to be implicit would be less ambiguous.

[snip]

 

It''s certain sanest as

    if x**2 + y**2 > 9 given x, y = func_returning_twople():

"given" really shines there!

Yep, that's why I don't have the same immediate reaction of "It would need to be limited to simple names as targets" reaction as I do for assignment expressions. It might still be a good restriction to start out with, though (especially if we wanted to allow multiple name bindings in a single given clause).

[snip]

The one-letter variable name obscures that it doesn't
actually reduce _redundancy_, though.  That is, in the current

    match = pattern.search(data)
    if match:

it's obviously less redundant typing as:

    if match := pattern.search(data):

In

    if match given match = pattern.search(data):

the annoying visual redundancy (& typing) persists.

Right, but that's specific to the case where the desired condition really is just "bool(target)". That's certainly likely to be a *common* use case, but if we decide that it's *that* particular flavour of redundancy that really bothers us, then there's always the "if expr as name:" spelling (similar to the way that Python had "a and b" and "a or b" logical control flow operators long before it got "a if c else b").

One more, a lovely (to my eyes) binding expression simplification
requiring two bindings in an `if` test, taken from real-life code I
happened to write during the PEP discussion:

    diff = x - x_base
    if diff:
        g = gcd(diff, n)
        if g > 1:
            return g

collapsed to the crisp & clear:

    if (diff := x - x_base) and (g := gcd(diff, n)) > 1:
        return g

If only one trailing "given" clause can be given per `if` test
expression, presumably I couldn't do that without trickery.

I was actually thinking that if we did want to allow multiple assignments, and we limited targets to single names, we could just use a comma as a separator:

    if diff and g > 1 given diff = x - x_base, g = gcd(diff, n):
        return g

Similar to import statements, optional parentheses could be included in the grammar, allowing the name bindings to be split across multiple lines:

    if diff and g > 1 given (
        diff = x - x_base,
        g = gcd(diff, n),
    ):
        return g

(Other potential separators would be ";", but that reads weirdly to me since my brain expects the semi-colon to end the entire statement, and "and", but that feels overly verbose, while also being overly different from its regular meaning)

 

  If it's
more general,

    if (diff given diff = x _ xbase) and g > 1 given g = gcd(diff, n):

reads worse to my eyes (perhaps because of the "visual redundancy"
thing again), while

   if diff and g > 1 given diff = x - x_base given g = gcd(diff, n):

has my eyes darting all over the place, and wondering which of the
trailing `given` clauses executes first.

I find that last effect is lessened when using the comma as a separator within the given clause rather than repeating the keyword itself.

[Nick Coghlan]

On 4 May 2018 at 22:06, Nick Coghlan <ncog...@gmail.com> wrote:

(Note: Guido's already told me off-list that he doesn't like the way this spelling reads, but I wanted to share it anyway since it addresses one of the recurring requests in the PEP 572 discussions for a more targeted proposal that focused specifically on the use cases that folks had agreed were reasonable potential use cases for inline assignment expressions.

I'll also note that another potential concern with this specific proposal is that even though "given" wasn't used as a term in any easily discovered Python APIs back when I first wrote PEP 3150, it's now part of the Hypothesis testing API, so adopting it as a keyword now would be markedly more disruptive than it might have been historically)

Since I genuinely don't think this idea is important enough to disrupt hypothesis's public API, I've been pondering potential synonyms that are less likely to be common in real world code, while still being comprehensible and useful mnemonics for the proposed functionality.

The variant I've most liked is the word "letting" (in the sense of "while letting these names have these values, do ..."):

    # Exactly one branch is executed here

    if m letting m = pattern.search(data):
        ...

    elif m letting m = other_pattern.search(data)):
        ...

    else:
        ...

    # This name is rebound on each trip around the loop

    while m letting m = pattern.search(remaining_data):
        ...

    # "f(x)" is only evaluated once on each iteration

    result = [(x, y, x/y) for x in data if y letting y = f(x)]

    # Tim's "bind two expressions" example

    if diff and g > 1 letting diff = x - x_base, g = gcd(diff, n):
        return g

    # The "bind two expressions" example across multiple lines

    while diff and g > 1 letting (

        diff = x - x_base,
        g = gcd(diff, n),
    ):

        ... # Do something with diff and g

[Nick Coghlan]

On 6 May 2018 at 02:06, Nick Coghlan <ncog...@gmail.com> wrote:

On 4 May 2018 at 22:06, Nick Coghlan <ncog...@gmail.com> wrote:

(Note: Guido's already told me off-list that he doesn't like the way this spelling reads, but I wanted to share it anyway since it addresses one of the recurring requests in the PEP 572 discussions for a more targeted proposal that focused specifically on the use cases that folks had agreed were reasonable potential use cases for inline assignment expressions.

I'll also note that another potential concern with this specific proposal is that even though "given" wasn't used as a term in any easily discovered Python APIs back when I first wrote PEP 3150, it's now part of the Hypothesis testing API, so adopting it as a keyword now would be markedly more disruptive than it might have been historically)

Since I genuinely don't think this idea is important enough to disrupt hypothesis's public API, I've been pondering potential synonyms that are less likely to be common in real world code, while still being comprehensible and useful mnemonics for the proposed functionality.

I've also been pondering Tim's suggestion of instead enhancing the code generation pipeline's native support for pseudo-keywords in a way that can be explicitly represented in the Grammar and AST, rather than having to be hacked in specifically every time we want to introduce a new keyword without a mandatory __future__ statement (and without creating major compatibility headaches for code that uses those new keywords as attribute or variable names).

While I haven't actually tried this out yet, the way I'm thinking that might look is to add the following nodes to the grammar:

    name_plus: NAME | pseudo_keyword

    pseudo_keyword: 'given'

and the replace all direct uses of 'NAME' in the grammar with 'name_plus'.

That way, to allow a new keyword to be used as a name in addition to its syntactic use case, we'd add it to the pseudo_keyword list in addition to adding it to .

To avoid ambiguities in the grammar, this could only be done for keywords that *can't* be used to start a new expression or statement (so it wouldn't have been sufficient for the async/await case, since 'async' can start statements, and 'await' can start both statements and expressions).

So if Guido's view on the out-of-order execution approach to inline name binding softens, I think this would be a better approach to pursue than making a more awkward keyword choice.

Cheers,

Nick.

[Tim Peters]

[Nick]
>...

> There were a couple key reasons I left the "for x in y" case out of the
> initial proposal:
>
> 1. The "for x in y" header is already quite busy, especially when tuple
> unpacking is used in the assignment target
> 2. Putting the "given" clause at the end would make it ambiguous as to
> whether it's executed once when setting up the iterator, or on every
> iteration
> 3. You can stick in an explicit "if True" if you don't need the given
> variable in the filter condition
>
> [(fx**2, fx**3) for x in xs if True given fx = f(x)]
>
> And then once you've had an entire release where the filter condition was
> mandatory for the comprehension form, allowing the "if True" in "[(fx**2,
> fx**3) for x in xs given fx = f(x)]" to be implicit would be less ambiguous.

And some people claim ":=" would make Python harder to teach ;-)


[Tim]
>> ...

>> It''s certain sanest as
>>
>> if x**2 + y**2 > 9 given x, y = func_returning_twople():
>>
>> "given" really shines there!

> Yep, that's why I don't have the same immediate reaction of "It would need
> to be limited to simple names as targets" reaction as I do for assignment
> expressions. It might still be a good restriction to start out with, though

I contrived that specific "use case", of course - I actually didn't
stumble into any real code where multiple targets would benefit to my
eyes. Perhaps because, as you noted above of `"for x in y" headers`,
multiple-target assignment statements are often quite busy already too
(I have no interest in cramming as much logic as possible into each
line - but "sparse is better than dense" doesn't also mean "almost
empty is better than sparse" ;-) ).

> (especially if we wanted to allow multiple name bindings in a single given
> clause).

>> ...

>> The one-letter variable name obscures that it doesn't
>> actually reduce _redundancy_, though. That is, in the current
>>
>> match = pattern.search(data)
>> if match:
>>
>> it's obviously less redundant typing as:
>>
>> if match := pattern.search(data):
>>
>> In
>>
>> if match given match = pattern.search(data):
>>
>> the annoying visual redundancy (& typing) persists.

> Right, but that's specific to the case where the desired condition really is
> just "bool(target)".

Not only. If the result _needs_ to be used N times in total in the
test, binding expressions allow for that, but `given` requires N+1
instances of the name (the "extra one" to establish the name to begin
with).

For example, where `probable_prime()` returns `True` or `False`, and
`bool(candidate)` is irrelevant:

# highbit is a power of 2 >= 2; create a random prime
# whose highest bit is highbit

while (not probable_prime(candidate) given candidate =
highbit | randrange(1, highbit, 2)):
pass

versus

while not probable_prime(candidate :=
highbit | randrange(1, highbit, 2)):
pass

There I picked a "long" name to make the redundancy visually annoying ;-)

> That's certainly likely to be a *common* use case,

In all the code I looked at where I believe a gimmick like this would
actually help, it was indeed by far _most_ common that the result only
needed to be used once in the test. In all such cases, the binding
expression spelling of the test requires one instance of the name, and
the `given` spelling two.

> but if we decide that it's *that* particular flavour of redundancy that really
> bothers us, then there's always the "if expr as name:" spelling (similar to
> the way that Python had "a and b" and "a or b" logical control flow
> operators long before it got "a if c else b").

Reducing each redundancy is a small win to me, but reaches
"importance" because it's so frequent. Binding expressions have more
uses than _just_ that, though.

But I'm sure I don't know what they all are. When a _general_ feature
is added, people find surprising uses for it.

For example, at times I'd love to write code like this, but can't:

while any(n % p == 0 for p in small_primes):
# divide p out - but what is p?

Generator expressions prevent me from seeing which value of `p`
succeeded. While that's often "a feature", sometimes it's a PITA. I
don't know whether this binding-expression stab would work instead
(I'm not sure the PEP realized there's "an issue" here, about the
intended scope for `thisp`):

while any(n % (thisp := p) == 0 for p in small_primes):
n //= thisp

If that is made to work, I think that counts as "a surprising use"
(capturing a witness for `any(genexp)` and a counterexample for
`all(genexp)`, both of which are wanted at times, but neither of which
`any()`/`all()` will ever support on their own)..

I suppose I could do it with `given` like so:

while p is not None given p = next(
(p for p in small_primes if n % p == 0),
None):
n //= p

but at that point I'd pay to go back to the original loop-and-a-half ;-)

>> One more, a lovely (to my eyes) binding expression simplification
>> requiring two bindings in an `if` test, taken from real-life code I
>> happened to write during the PEP discussion:
>>
>> diff = x - x_base
>> if diff:
>> g = gcd(diff, n)
>> if g > 1:
>> return g
>>
>> collapsed to the crisp & clear:
>>
>> if (diff := x - x_base) and (g := gcd(diff, n)) > 1:
>> return g
>>
>> If only one trailing "given" clause can be given per `if` test
>> expression, presumably I couldn't do that without trickery.

> I was actually thinking that if we did want to allow multiple assignments,
> and we limited targets to single names, we could just use a comma as a
> separator:
>
> if diff and g > 1 given diff = x - x_base, g = gcd(diff, n):
> return g

I expect that 's bound to be confusing, because the assignment _statement_

diff = x - x_base, g = gcd(diff, n)

groups very differently than intended:

diff = (x - x_base, g) = gcd(diff, n)

And that's a syntax error. With enclosing parens, expectations
change, and then people would expect it to work like specifying
keyword arguments instead:

> Similar to import statements, optional parentheses could be included in the
> grammar, allowing the name bindings to be split across multiple lines:
>
> if diff and g > 1 given (
> diff = x - x_base,
> g = gcd(diff, n),
> ):
> return g

Keyword arguments work as a syntactic model (group as intended), but
not semantically: if they really were keyword arguments, `x - x_base`
and `gcd(diff, n)` would both be evaluated _before_ any bindings
occurred.

So it's more quirky `given`-specific rules no matter how you cut it.
The closest bit of Python syntax that captures the grouping (but only
partially), and the "left-to-right, with each binding in turn visible
to later expressions" semantics, is the semicolon. Which would create
even weirder expectations :-(

> (Other potential separators would be ";", but that reads weirdly to me since
> my brain expects the semi-colon to end the entire statement, and "and", but
> that feels overly verbose, while also being overly different from its
> regular meaning)

Yup.

>> If it's more general,
>>
>> if (diff given diff = x _ xbase) and g > 1 given g = gcd(diff, n):
>>
>> reads worse to my eyes (perhaps because of the "visual redundancy"
>> thing again), while
>>
>> if diff and g > 1 given diff = x - x_base given g = gcd(diff, n):
>>
>> has my eyes darting all over the place, and wondering which of the
>> trailing `given` clauses executes first.

> I find that last effect is lessened when using the comma as a separator
> within the given clause rather than repeating the keyword itself.

Definitely. I tend to believe Python has "slightly more than enough"
meanings for commas already, though. But using commas and _requiring_
parens for more than one `given` binding seems least surprising to me
overall.

Then again, everyone already knows what ":=" means. They just dislike
it because so many major languages already have it -)

[Brett Cannon]

My brain wants to drop the variable name in front of 'given':

if given m = pattern.search(data):

while given m = pattern.search(remaining_data):

Maybe it's because the examples use such a short variable name?

if match given match = pattern.search(data):

vs.

if given match = pattern.search(data);

Nope, I still like mine more. ;)

-Brett

 

Constraining the syntax that way (at least initially) would avoid poking into any dark corners of Python's current scoping and expression execution ordering semantics, while still leaving the door open to later making "result given NAME = expr" a general purpose ternary operator that returns the LHS, while binding the RHS to the given name as a side effect.

Using a new keyword (rather than a symbol) would make the new construct easier to identify and search for, but also comes with all the downsides of introducing a new keyword. (Hence the not-entirely-uncommon suggestion of using "with" for a purpose along these lines, which runs into a different set of problems related to trying to use "with" for two distinct and entirely unrelated purposes).

Cheers,

Nick.

--

Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia

[Nick Coghlan]

On 8 May 2018 at 04:19, Brett Cannon <br...@python.org> wrote:

My brain wants to drop the variable name in front of 'given':

if given m = pattern.search(data):

while given m = pattern.search(remaining_data):

Maybe it's because the examples use such a short variable name?

Does that change if the condition isn't just "bool(name)"? For example:

    if y > 0 given y = f(x):
        ...

That's the situation where I strongly prefer the postfix operator spelling, since it's pretty clear how I should pronounce it (i.e. "if y is greater than zero, given y is set to f-of-x, then ..."). By contrast, while a variety of  plausible suggestions have been made, I still don't really know how to pronounce "if (y := f(x)) > 0:)" in a way that's going to be clear to an English-speaking listener (aside from pronouncing it the same way as I'd pronounce the version using "given", but that then raises the question of "Why isn't it written the way it is pronounced?").

I do agree with Tim that the name repetition would strongly encourage the use of short names rather than long ones (since you're always typing them at least twice), such that we'd probably see code like:

    while not probable_prime(n) given (n =

                            highbit | randrange(1, highbit, 2)):
        pass

Rather than the more explicit:

    while not probable_prime(candidate) given (candidate =

                            highbit | randrange(1, highbit, 2)):
        pass

However, I'd still consider both of those easier to follow than:

    while not probable_prime(candidate := highbit | randrange(1, highbit, 2)):
        pass

since it's really unclear to me that "candidate" in the latter form is a positional argument being bound to a name in the local environment, and *not* a keyword argument being passed to "probable_prime".

I've also been pondering what the given variant might look like as a generally available postfix operator, rather than being restricted to if/elif/while clauses, and I think that would have interesting implications for the flexibility of its usage in comprehensions, since there would now be *three* places where "given" could appear (as is already the case for the inline binding operator spelling):

- in the result expression

- in the iterable expression

- in the filter expression

That is:

    [(x, y, x - y) given y = f(x) for x in data]

    [(x, data) for x in data given data = get_data()]

    [(x, y, x/y) for x in data if y given y = f(x)]

Rather than:

    [(x, y := f(x), x - y) for x in data]

    [(x, data) for x in data := get_data()]

    [(x, y, x/y) for x in data if y := f(x)]

Opening it up that way would allow for some odd usages that might need to be discouraged in PEP 8 (like explicitly preferring "probable_prime(n) given n = highbit | randrange(1, highbit, 2)" to "probable_prime(n given n = highbit | randrange(1, highbit, 2))"), but it would probably still be simpler overall than attempting to restrict the construct purely to if/elif/while.

Even as a generally available postfix keyword, "given" should still be amenable to the treatment where it could be allowed as a variable name in a non-operator context (since we don't allow two adjacent expressions to imply a function call, it's only prefix keywords that have to be disallowed as names to avoid ambiguity in the parser).

Cheers,

Nick.

[Guido van Rossum]

I'm sorry, but unless there's a sudden landslide of support for 'given' in favor of ':=', I'm really not going to consider it.

I'd pronounce "if (x := y) > 0" as either "if y (assigned to x) is greater than zero" or "if x (assigned from y) is greater than zero".

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Rhodri James]

On 10/05/18 14:44, Guido van Rossum wrote:
> I'm sorry, but unless there's a sudden landslide of support for 'given' in
> favor of ':=', I'm really not going to consider it.

OK, this is my ha'p'th in favour of 'given', for what little it's worth.
The more I see of general assignment expressions, the less I like
them. All I really want is a less clumsy way to write

while true:
thing_to_do = get_something_to_do()
if thing_to_do == GET_OUT_OF_HERE:
break
# else do stuff

--
Rhodri James *-* Kynesim Ltd

[marky1991 .]

If it just needs a stream of +1s, I personally like the "given" approach much more than the ":=" approach, for all of the many reasons repeated many times in the various email chains. (I preferred it as "as", but that's been struck down already) (and if it's between ":=" and not having them at all, I would rather just not have them)

[Guido van Rossum]

Please no, it's not that easy. I can easily generate a stream of +1s or -1s for any proposal. I'd need well-reasoned explanations and it would have to come from people who are willing to spend significant time writing it up eloquently. Nick has tried his best and failed to convince me. So the bar is high.

(Also note that most of the examples that have been brought up lately were meant to illustrate the behavior in esoteric corner cases while I was working out the fine details of the semantics. Users should use this feature sparingly and stay very far away of those corner cases -- but they have to be specified in order to be able to implement this thing.)

On Thu, May 10, 2018 at 10:26 AM, marky1991 . <mark...@gmail.com> wrote:

If it just needs a stream of +1s, I personally like the "given" approach much more than the ":=" approach, for all of the many reasons repeated many times in the various email chains. (I preferred it as "as", but that's been struck down already) (and if it's between ":=" and not having them at all, I would rather just not have them)

[Ed Kellett]

On 2018-05-10 16:10, Guido van Rossum wrote:
> Please no, it's not that easy. I can easily generate a stream of +1s or -1s
> for any proposal. I'd need well-reasoned explanations and it would have to
> come from people who are willing to spend significant time writing it up
> eloquently. Nick has tried his best and failed to convince me. So the bar
> is high.
>
> (Also note that most of the examples that have been brought up lately were
> meant to illustrate the behavior in esoteric corner cases while I was
> working out the fine details of the semantics. Users should use this
> feature sparingly and stay very far away of those corner cases -- but they
> have to be specified in order to be able to implement this thing.)

Poor prospects, then, but I'll do my best.

I think the most obvious argument (to me) favouring `given` over `:=` is
that it separates the two things it's doing:

if m.group(2) given m = pattern.search(data):

as opposed to the more-nested := version:

if (m := pattern.search(data)).group(2):

which, at least to me, is more complicated to think about because it
feels like it's making the .group() something to do with the assignment.

Put another way, I think your use of parentheses when discussing the
*pronunciation* of this thing is telling. It feels as though one needs
to start in the middle and then go in both directions at once, first
explaining the origin (or destination) of the operand in question in a
parenthesized offshoot, and then switching context and describing what
is done to it. It's midly mentally taxing. I'm sure we can all live with
that, but I don't want to: Python's exceptionally-readable syntax is one
of the bigger reasons I choose it.

There's a striking parallel in C, where the well-known idiom:

while ((c = getchar()) != EOF) ...

has an obviously-nicer alternative:

while (c = getchar(), c != EOF) ...

Most people I show this to agree that it's nicer, despite the fact that
it manages to repeat a variable name *and* use the comma operator. I
don't have proof, but I'd suggest that unwrapping that layer of context
for the reader imparts a significant benefit.

The C example also provides a convenient test: if you think the former
example is nicer, I can just give up now ;)

[Gustavo Carneiro]

IMHO, all these toy examples don't translate well to the real world because they tend to use very short variable names while in real world [good written code] tends to select longer more descriptive variable names.

Try replacing "c" with  a longer name, like input_command, then it becomes:

    while ((input_command = getchar()) != EOF) ...

    while (input_command = getchar(), input_command != EOF) ...

In the second example, having to type the variable name twice is an  annoyance that adds almost nothing to readability, so I would definitely prefer the first one.

The "given" proposals have the same issue.  

(a shame we can't use "as", for reasons already stated, it would have been perfect otherwise)

--

Gustavo J. A. M. Carneiro

Gambit Research
"The universe is always one step beyond logic." -- Frank Herbert

[Alexander Belopolsky]

On Thu, May 10, 2018 at 9:44 AM, Guido van Rossum <gu...@python.org> wrote:
> I'm sorry, but unless there's a sudden landslide of support for 'given' in
> favor of ':=', I'm really not going to consider it.

How much support was there for ":="? Are you serious about bringing
back Pascal and Algol from their comfortable resting places?

[Guido van Rossum]

Yes.

[Ryan Gonzalez]

Probably going to completely lose this, but would it be possible to have a vote? +1 for either 'given' and/or ':='? 

_______________________________________________

Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--

Ryan (ライアン)
Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else
https://refi64.com/

[David Mertz]

Only the BDFL has a vote with non-zero weight.

But I'll contribute my zero-weighted preference for :=.

[Terry Reedy]

On 5/10/2018 9:44 AM, Guido van Rossum wrote:
> I'm sorry, but unless there's a sudden landslide of support for 'given'
> in favor of ':=', I'm really not going to consider it.
>
> I'd pronounce "if (x := y) > 0" as either "if y (assigned to x) is
> greater than zero" or "if x (assigned from y) is greater than zero".

or "if x (bound to y) is greater than zero" ("do something with x").

--
Terry Jan Reedy

[Kirill Balunov]

2018-05-10 16:44 GMT+03:00 Guido van Rossum <gu...@python.org>:

I'm sorry, but unless there's a sudden landslide of support for 'given' in favor of ':=', I'm really not going to consider it.

I'd pronounce "if (x := y) > 0" as either "if y (assigned to x) is greater than zero" or "if x (assigned from y) is greater than zero".

I think you do not quite objectively look at the current situation. Many just lost interest in attempts to move the topic at least a little bit in the other way, seeing how you and Tim so actively expresses support/protects this `:=` syntax, while ignoring or pushing out alternative opinions :-). Of course, the latter is partly due to the incredible number of different threads and messages on this topic.

 

Briefly:

Initially, the main argument in favor of `:=` was that this form is similar to the usual assignment statement, but can be used as an expression. Ok.

Then everyone agreed with the idea that it's necessary to limit assignment target to name only. Although all this criticism was actually put forward in the first 50-100 messages on the topic.

In the same first hundred it was actively discussed, that in fact, this idea gives a win only in `while` and `if` statemetns that probably will match 99%+ where it will be used for its intended purpose. At the same time, most of the criticism concerned exactly the use in generators and comprehenshions, they are already often overloaded for perception. And as you once said - "Language Design Is Not Just Solving Puzzles".

There was also discussed the difference in perception between the `expr op name` and `name op expr`. Here the expression is something that is important, the name is only a convenient consequence.

At the moment with all the constraints of `:=`, the discuscation is more like - trying to cram this syntax into the language. While for those who are familiar with Pascal, Icon and other languages that use this syntax, this - `:=` looks natural. For others and I believe such a majority among users, this syntax is, to put it mildly, not natural and ugly producing a line noise, the colon `:` symbol is already used in a lot of places.

With all the changes, the limitations and magic with scopes. Is it now easier to explain all the differences between `=` and `:=`, than the difference between `if expr as name ...: ...` and `with expr as name:`?

Therefore, I take Nick's insistent position as an attempt to at least somehow make an alternative look at this topic.

With kind regards,

-gdg

[Nick Coghlan]

On 10 May 2018 at 11:10, Guido van Rossum <gu...@python.org> wrote:

Please no, it's not that easy. I can easily generate a stream of +1s or -1s for any proposal. I'd need well-reasoned explanations and it would have to come from people who are willing to spend significant time writing it up eloquently. Nick has tried his best and failed to convince me. So the bar is high.

(Also note that most of the examples that have been brought up lately were meant to illustrate the behavior in esoteric corner cases while I was working out the fine details of the semantics. Users should use this feature sparingly and stay very far away of those corner cases -- but they have to be specified in order to be able to implement this thing.)

I raised this with some of the folks that were still here at the Education Summit (similar to what I did for data classes at the PyCon Australia education seminar last year), but whereas the reactions to data classes were "as easy or easier to teach as traditional classes", the reaction for this for the folks that I asked was almost entirely negative - the most positive reaction was "Yes, if it's as a wholesale replacement for the '=' spelling, since that sometimes gets confused with mathematical equality". As an *addition* to the existing spelling, and especially with the now proposed leaking semantics in comprehension scopes, it was "No, that would just confuse out students".

It's one thing adding syntactic and semantic complexity for the sake of something that significantly increases the language's expressive power (which is what the original sublocal scopes proposal was aiming for: the ability to more readily express constrained scoping and name shadowing without explicit name aliasing and del statements), it's something else entirely to do it for the sake of purely cosmetic tweaks like flattening the occasional nested if-else chain or replacing a loop-and-a-half with an embedded assignment.

Regards,

Nick.

[Nick Malaguti]

Hi all. I've been lurking for a little while on this discussion and I thought I might contribute some thoughts.

One of my hurdles for ":=" is understanding when I should use it rather than "=". Should I use it everywhere? Should I use it only where I can't use regular "="? Is it a personal choice? Will it become so common that I need to think harder because some people will use it really frequently or intermix them?

I don't want to see "=" vs ":=" become like semicolons in JavaScript. When I work on a different codebase, am I going to have to follow an "always" or "never" for binding expressions? Maybe this is all overblown and PEP8 direction will keep everyone on the same page, but I guess I worry about there being 2 very similar, but not the same, ways to do it.

What I really like about "given" is it makes it a lot clearer when I should use it. No one is going to want to write

x given x = f()

if they can just write

x = f()

If you need a binding expression in a comprehension or an if or while statement, you'll know the pattern of using "given" to save that loop and a half or to call a function and bind its result while iterating. Just like you know when to use a ternary if to save that extra temporary variable - there's little confusion about when to use a ternary, especially since a few if statements quickly prove clearer to read.

10 if x == 5 else 9 if x == 2 else 8 if x == 3 else 100

looks much better as:

if x == 5:

    result = 10

elif x == 2:

    result = 9

elif x == 3:

    result = 8

else:

    result = 100

I feel the same way about given. If you feel tempted to go overboard with:

x given x = y * 2 given y = z + 3 given z = f()

Which should be equivalent to:

x := (y := ((z := f()) + 3)) * 2

hopefully you'll think, "maybe I should just make 3 statements instead?"

And also I have no trouble following what that statement actually does when using given. I didn't need any parenthesis to make sure I didn't bind the wrong expressions and I don't have to read it from the inside out. Each sub-expression is complete rather than being mixed together (even though I have to read it from right to left).

I feel like the strongest argument for ":=" is for all the situations where someone will actually want a binding expression in real code, ":=" is more succinct. I'm just concerned that when given a new binding expression hammer, everything is going to look like a nail and all the places where someone could really benefit from a binding expression will be drowned out by the unnecessary usage of ":=" (and its side effects).

--

Nick

[Greg Ewing]

+1 given, -1 :=

--
Greg

[Greg Ewing]

Guido van Rossum wrote:
> I'd need well-reasoned explanations

My reasoning is essentially the same as what I've already
said about "where". To summarise, "given" sounds like
something an English-speaking mathematician would write,
whereas ":=" doesn't even have an obvious pronunciation.
Some variation on "given" just seems greatly more pythonic
to me.

--
Greg

[Greg Ewing]

Gustavo Carneiro wrote:
> IMHO, all these toy examples don't translate well to the real world
> because they tend to use very short variable names while in real world
> [good written code] tends to select longer more descriptive variable names.

I don't believe that's always true. It depends on the context.
Sometimes, using long variable names can make code *harder*
to read.

I don't think there's anything unrealistic about this
example:

if m given m = pattern.match(the_string):
nugget = m.group(2)

Most people's short-term memory is good enough to remember
that "m" refers to the match object while they read the
next couple of lines. IMO, using a longer name would serve
no purpose and would just clutter things up.

--
Greg

[Greg Ewing]

Kirill Balunov wrote:
>
> While for those who
> are familiar with Pascal, Icon and other languages that use this syntax,
> this - `:=` looks natural.

As someone familiar with Pascal, I think the similarity to
the Pascal assignment operator is actually an argument
*against* it. Knowing what it means in Pascal is confusing,
because Pascal's ":=" is equivalent to Python's "=" (it's
strictly a statement, and can't be used in expressions).

--
Greg

[João Santos]

+1 to this reasoning. One of the main reason python is popular is because code is easy to read, while ":=" would clearly not be as readable as "given". For me the difference between "given" and ":=" is the same as between python and C for loops.

[Jacco van Dorp]

I dont really like "given".

If we compare:

if m given m = re.match(stuff):

to

if m := re.match(stuff)

then I count 4+(name_length) more tokens and 2 more spaces. Since I
believe := is perfectly clear, I don't see the reason for a far more
verbose syntax.

That all said, I would still prefer:

if re.match(stuff) as m:

which is exactly equal to the := in line length and parallels with.
While that may -technically- be a different beast.
For beginners the difference is really irrelevant, and you can just
tell advanced people the full story(technically speaking the as in a
with statement isn't an expression assignment, it's a part of the with
statement, and it feeds in the value through the context manager
machinery before binding it. Similar for the except statement.).

But I've kind of given up on "as" (so no need to reply on that bit).

[João Santos]

Optimizing syntax for space makes sense for "mathematical" notation since it's commonly written by hand, but putting space above readability in a programming language design feels like a skewmorphism.

[Jacco van Dorp]

2018-05-11 11:56 GMT+02:00 João Santos <jm...@jsantos.eu>:
> Optimizing syntax for space makes sense for "mathematical" notation since
> it's commonly written by hand, but putting space above readability in a
> programming language design feels like a skewmorphism.

You are assuming "given" to improve readability, where I stated ":= is
perfectly clear ", at least in my opinion. Therefore, since clarity is
already achieved, the rest is clutter that reduces readability.

[Sven R. Kunze]

On 11.05.2018 09:33, Greg Ewing wrote:
> Gustavo Carneiro wrote:
>> IMHO, all these toy examples don't translate well to the real world
>> because they tend to use very short variable names while in real
>> world [good written code] tends to select longer more descriptive
>> variable names.
>
> I don't believe that's always true. It depends on the context.
> Sometimes, using long variable names can make code *harder*
> to read.
>
> I don't think there's anything unrealistic about this
> example:
>
>    if m given m = pattern.match(the_string):
>       nugget = m.group(2)

I gather we don't talk about interactive usage.

So, I grepped through part of our code-base. Like it or not, it's almost
always called "match" there.

Like Gustavo, I also have the feeling that long-living, real-world code
tends to have more descriptive names than all those toy examples/arguments.

Cheers,
Sven

[Ed Kellett]

On 2018-05-10 17:10, Gustavo Carneiro wrote:
> IMHO, all these toy examples don't translate well to the real world because
> they tend to use very short variable names while in real world [good
> written code] tends to select longer more descriptive variable names.
>
> Try replacing "c" with a longer name, like input_command, then it becomes:
>
> while ((input_command = getchar()) != EOF) ...
>
> while (input_command = getchar(), input_command != EOF) ...

This thread really isn't about variable naming. I'll simply say that
I've always written the C idiom with `c` as the variable name, I've seen
literally hundreds of others do the same, and I've never seen anyone
spell it `input_command`.

I do the same in Python in a few contexts, usually where the variable's
meaning is very clear from its usage, or where the code in question
doesn't know or care what the variable is used for:

for i in range(10):
def get_const(self, x):
for k in self.defaults.keys():

etc., and if we had `given` clauses, I'd probably do this there too. I
think one of the advantages of that syntax is that it makes it extremely
clear what's going on:

if m given m = re.match(...):

I don't need to try to stuff an English description of m into its name,
because the `given` clause already describes it perfectly.

[Sven R. Kunze]

On 11.05.2018 09:38, Greg Ewing wrote:
> Kirill Balunov wrote:
>>
>> While for those who are familiar with Pascal, Icon and other
>> languages that use this syntax, this - `:=` looks natural.
>
> As someone familiar with Pascal, I think the similarity to
> the Pascal assignment operator is actually an argument
> *against* it. Knowing what it means in Pascal is confusing,
> because Pascal's ":=" is equivalent to Python's "=" (it's
> strictly a statement, and can't be used in expressions).

Same here. It means something different.

Also coding in Pascal was annoying from the beginning with its extremely
verbose syntax like begin/end etc.
So, ":=" also felt like "why the hell do we need a colon in front of the
equal sign?"
Absolutely unnecessary bloat, like almost everything in Pascal.

Maybe that's also part, why I am -1 on the proposal. Who knows...

[Rhodri James]

On 11/05/18 11:14, Jacco van Dorp wrote:
> 2018-05-11 11:56 GMT+02:00 João Santos <jm...@jsantos.eu>:
>> Optimizing syntax for space makes sense for "mathematical" notation since
>> it's commonly written by hand, but putting space above readability in a
>> programming language design feels like a skewmorphism.
>
> You are assuming "given" to improve readability, where I stated ":= is
> perfectly clear ", at least in my opinion. Therefore, since clarity is
> already achieved, the rest is clutter that reduces readability.

I respectfully disagree with your opinion (i.e. you're wrong :-)

Consider:

while (cmd := get_command()).token != CMD_QUIT:
cmd.do_something()

vs:

while cmd.token != CMD_QUIT given cmd = get_command():
cmd.do_something()

I find I write code like this[*] a fair bit, since my major use for
Python is to write remote monitors for embedded kit, so it's pretty much
a real world example. I don't find the first version using ":=" to be
perfectly clear, in fact I think it's rather ugly. That may be partly
the same reaction that many of us had to the asymmetry of assignment
expressions in (over-)complicated comprehensions. The second version
using "given" reads much more naturally to the mathematician in me, and
not too badly to my English half either.

[*] By "like this" I mean the clunky "while true:" spelling, obviously.

--
Rhodri James *-* Kynesim Ltd

[Nick Coghlan]

On 11 May 2018 at 03:33, Greg Ewing <greg....@canterbury.ac.nz> wrote:

Gustavo Carneiro wrote:

IMHO, all these toy examples don't translate well to the real world because they tend to use very short variable names while in real world [good written code] tends to select longer more descriptive variable names.

I don't believe that's always true. It depends on the context.
Sometimes, using long variable names can make code *harder*
to read.

I don't think there's anything unrealistic about this
example:

   if m given m = pattern.match(the_string):
      nugget = m.group(2)

Most people's short-term memory is good enough to remember
that "m" refers to the match object while they read the
next couple of lines. IMO, using a longer name would serve
no purpose and would just clutter things up.

I've been thinking about this problem, and I think for the If/elif/while cases it's actually possible to allow the "binding is the same as the condition" case to be simplified to:

    if command =  pattern.match(the_string):
        ...
    elif command =  other_pattern.match(the_string):
        ...

    while data = read_data():
        ...

Allowing this would be part of the definition of the if/elif/while statement headers, rather than a general purpose assignment expression.

The restriction of the LHS to a simple name target would need to be in the AST generator rather than in the grammar, but it's hardly the only case where we do that kind of thing. Switching to the given expression form would then only be necessary in cases where the condition *wasn't* the same as the binding target.

A similar enhancement could be made to conditional expressions (adjusting their grammar to permit "EXPR if NAME = EXPR else EXPR") and filter clauses in comprehensions (allowing "EXPR for TARGET in EXPR if NAME = EXPR"). In essence, "if", "elif", and "while" would all allow for an "implied given" clause in order to simplify the 90% case where the desired condition and the bound expression are the same.

Cheers,

[Steven D'Aprano]

On Sat, May 05, 2018 at 06:24:07AM -0400, Juancarlo Añez wrote:
> I think that "expr as y" was discarded too quickly.

This discussion started back in *February*. I don't think "too quickly"
applies to ANYTHING about it.

https://mail.python.org/pipermail/python-ideas/2018-February/048971.html

And Chris' first draft of the PEP:

https://mail.python.org/pipermail/python-ideas/2018-February/049041.html

I have been one of the main proponents of "as". See, for example:

https://mail.python.org/pipermail/python-ideas/2018-April/049880.html

At least, I *was*. I'm now satisfied that "as" is the wrong solution,
and I don't think it was discarded too quickly. Even though it makes me
sad that "as" is not suitable, I'm satisfied that the problems with it
would require too high a price to solve.

The major problem is that it will clash with "except as" and "with as"
statements. Of course we *could* introduce some sort of special
treatment, possibly as simple as simply banning the use of binding-
assignments inside except/with statements, but such special rules add
complexity, make the feature less useful, harder to learn, and more
surprising.

Allowing or disallowing particular expressions after a certain keywork
ought to be a last resort. Even though this was my preferred solution,
I've now come to change my mind and think this would have been a
mistake.

(Thanks Chris for sticking to your guns and rejecting "as".)

You say:

> The special cases that may arise over "except" and "with" can be worked
> out and documented.

but there's no "may" about this. Using "as" does clash, it's not a
matter of whether or not it will clash, we know it will. And it's easy
to say that it "can" be worked out, but unless you have a concrete
proposal to work it out, that's not really an argument in favour for
"as", it is just a hope.

Guido has also correctly pointed out that will "as" is used to bind
names in other contexts, it doesn't *quite* work the same as regular =
assignment. Again, the "with" statement is especially relevant:

with expression as name

does not bind the value of the expression to name, except by
coincidence. It actually binds the value of expession.__enter__() to
name.

I still, and probably always will, like the look of

result = (expression as spam) + spam**2

but I'm realistic to realise that it isn't practical.



--
Steve

[Sven R. Kunze]

On 11.05.2018 13:43, Nick Coghlan wrote:

I've been thinking about this problem, and I think for the If/elif/while cases it's actually possible to allow the "binding is the same as the condition" case to be simplified to:

    if command =  pattern.match(the_string):
        ...
    elif command =  other_pattern.match(the_string):
        ...

    while data = read_data():
        ...

Allowing this would be part of the definition of the if/elif/while statement headers, rather than a general purpose assignment expression.

I can imagine that this will cover 80 to 90% of the usecases and it's readable as well.

A similar enhancement could be made to conditional expressions (adjusting their grammar to permit "EXPR if NAME = EXPR else EXPR") and filter clauses in comprehensions (allowing "EXPR for TARGET in EXPR if NAME = EXPR").

Not sure if that is too much for now. List comprehensions tend to be longer than expected, the same goes for the ternary expression.

Maybe, we could start with the 90% case and whether the need for more arises.

In essence, "if", "elif", and "while" would all allow for an "implied given" clause in order to simplify the 90% case where the desired condition and the bound expression are the same.

Exactly. Maybe, even here: let's do just the 90% case. And if a lot of people need finer control, we can reconsider :=/given/as.


Regards,
Sven

[Steven D'Aprano]

On Fri, May 04, 2018 at 09:56:10PM -0400, Alexander Belopolsky wrote:

> This proposal has finally made me realize why I did not
> like PEP 572. The strong expression vs. statement dichotomy is one of
> the key features that set Python apart from many other languages and
> it makes Python programs much easier to understand.

I'm not so sure that the "expression versus statement" dichotomy is as
strong or as useful as Alexander says. If it were, we'd be writing much
more imperative code, as if it were 1970 and we were using BASIC.

Some of the greatest Python successes have been to add expression
forms of what used to be purely imperative statements:

- comprehensions (for-loops);

- ternary if (if...else statement).

In addition, we have more overlap between statements and expressions:

- an expression form of def (lambda);

- a functional form of the import statement (importlib.import_module,
before that people used to use __import__();

- expression forms of augmented assignment (ordinary + etc operators),
which is a rare case where the expression form came first and the
imperative command came afterwards.

I may have missed some.

Also notable is that Python does not have "procedures" (pure statement
callables). We use functions instead, and simply ignore the returned
result.

If the difference between statements and expressions was really a
dichotomy, we would only need One Way to do these things, not two.

I also have seen many people disappointed that Python doesn't treat
"everything as an expression". The lack of assignment-expressions is a
turn-off for some people:

https://mail.python.org/pipermail/python-list/2018-May/732890.html

See also:

https://stackoverflow.com/questions/50090868/why-are-assignments-not-allowed-in-pythons-lambda-expressions

[Chris Angelico]

On Fri, May 11, 2018 at 9:37 PM, Rhodri James <rho...@kynesim.co.uk> wrote:
> On 11/05/18 11:14, Jacco van Dorp wrote:
>>
>> 2018-05-11 11:56 GMT+02:00 João Santos <jm...@jsantos.eu>:
>>>
>>> Optimizing syntax for space makes sense for "mathematical" notation since
>>> it's commonly written by hand, but putting space above readability in a
>>> programming language design feels like a skewmorphism.
>>
>>
>> You are assuming "given" to improve readability, where I stated ":= is
>> perfectly clear ", at least in my opinion. Therefore, since clarity is
>> already achieved, the rest is clutter that reduces readability.
>
>
> I respectfully disagree with your opinion (i.e. you're wrong :-)
>
> Consider:
>
> while (cmd := get_command()).token != CMD_QUIT:
> cmd.do_something()
>
> vs:
>
> while cmd.token != CMD_QUIT given cmd = get_command():
> cmd.do_something()
>

Yes, I've considered it. And I don't like the fact that the evaluation
is right-to-left. It isn't a problem when your condition is extremely
simple, but I can guarantee you that people will use this with more
complicated conditions. And when that happens, you have to be aware
that the tail of the statement is actually evaluated before the
primary expression. It's like with Perl:

die("blah blah") unless some_condition

Reverse-order evaluation is confusing and frequently annoying. It's
not an instant failure of the proposal, but it's a serious cost, and
I'd much rather avoid it by using := (which leaves the expression
where it is).

ChrisA

[Jacco van Dorp]

>[Rhodri]

> I respectfully disagree with your opinion (i.e. you're wrong :-)
>
> Consider:
>
> while (cmd := get_command()).token != CMD_QUIT:
> cmd.do_something()
>
> vs:
>
> while cmd.token != CMD_QUIT given cmd = get_command():
> cmd.do_something()
>

Actually, the first version is more readable. It's got a lot to do
with what Chris said about order of operations, but IMO, even more
with grouping. There's a couple of things you might want to learn from
this statement. First, will the while check succeed? Well....


while......get_command()).token != CMD_QUIT:

yup, looks clear. I can ignore the cmd := part easily, and the bonus
paren I see there doesn't matter that much.

Another thing i might be curious about, is what is the value of cmd after ?

while (cmd := get_command())........................:

Looks like it has the value of getcommand().

Hey, that was both clear and readable. I can just ignore half the line
and learn stuff. Great.

Lets see with the other notation. What's the value of cmd ?

while ................................................... cmd = get_command():

That works. Bit of line I had to skip.

Will the check succeed ?

while cmd.token != CMD_QUIT .....................................:

Wait, what's the value of cmd ? Lets look in the code in the preceding
lines....oh, ok, it's at the end of the line.

I actually have to mentally parse the entire line to get what the
check will work. This, along with what Chris said about order of
operations, reduce the readability of the "given" version.

[Rhodri James]

On 11/05/18 14:25, Jacco van Dorp wrote:
> I actually have to mentally parse the entire line to get what the
> check will work. This, along with what Chris said about order of
> operations, reduce the readability of the "given" version.

You say "I had to parse the entire line..." I hear "I had to read what
I was making a snap decision on." Sounds like a win :-)

--
Rhodri James *-* Kynesim Ltd

[Chris Angelico]

On Fri, May 11, 2018 at 11:34 PM, Rhodri James <rho...@kynesim.co.uk> wrote:
> On 11/05/18 14:25, Jacco van Dorp wrote:
>>
>> I actually have to mentally parse the entire line to get what the
>> check will work. This, along with what Chris said about order of
>> operations, reduce the readability of the "given" version.
>
>
> You say "I had to parse the entire line..." I hear "I had to read what I
> was making a snap decision on." Sounds like a win :-)
>

No, not a win. Do you read the entire source code for an entire
project before trying to comprehend one part of it? I doubt it. Do you
read an entire file before trying to comprehend a single function in
that file? No. Do you even read an entire function before processing
one line in that function? Unlikely. It's normal and correct to seek
to understand one part of some code while ignoring other parts. That's
why we have proper variable names, even inside functions - we could
just use "slot0" and "slot1" and so on, since that's how they work to
the interpreter. But we use good names, so that you can understand
some code without having to first read the thing that created that
variable.

ChrisA

[João Santos]

How do you read something like " while (cmd := get_command()).token != CMD_QUIT:" in plain english?

[Jacco van Dorp]

A while ago, we had this gem:

2018-04-06 8:19 GMT+02:00 Serhiy Storchaka <stor...@gmail.com>:
> Using currently supported syntax:
>
> smooth_signal = [average for average in [0] for x in signal for average in [(1-decay)*average + decay*x]]

Go ahead and understand that line in 1 go. It's currently legal syntax
for a running average for a smoothing signal, which remembers
something about it. (Subject: Proposal: A Reduce-Map Comprehension and
a "last" builtin)

You're not allowed to work it out bit by bit, just understand the
entire line or nothing. Any failure of yours proves my point.

> [João]
> How do you read something like " while (cmd := get_command()).token != CMD_QUIT:" in plain english?

while open-paren cee em dee colon is call get-underscore-command
close-paren dot token doesn't equal all-caps cee em dee underscore
quit colon.

Might be some dutch in there.

But far more importantly, I can hold the concept into my head, or just
the parts of it that I need. How we call it in english is actually not
a good argument - whether we can easily mentally parse it is, since I
tend not to code by voice command, but with a keyboard. Your mileage
may vary, but I think we should optimize for keyboard coding over
voice chat coding. And when I need to refer to it, I say "this bit
here" or I copy paste it.

[Clint Hepner]

> On 2018 May 11 , at 7:37 a, Rhodri James <rho...@kynesim.co.uk> wrote:
>
> On 11/05/18 11:14, Jacco van Dorp wrote:
>> 2018-05-11 11:56 GMT+02:00 João Santos <jm...@jsantos.eu>:
>>> Optimizing syntax for space makes sense for "mathematical" notation since
>>> it's commonly written by hand, but putting space above readability in a
>>> programming language design feels like a skewmorphism.
>> You are assuming "given" to improve readability, where I stated ":= is
>> perfectly clear ", at least in my opinion. Therefore, since clarity is
>> already achieved, the rest is clutter that reduces readability.
>
> I respectfully disagree with your opinion (i.e. you're wrong :-)
>
> Consider:
>
> while (cmd := get_command()).token != CMD_QUIT:
> cmd.do_something()
>
> vs:
>
> while cmd.token != CMD_QUIT given cmd = get_command():
> cmd.do_something()
>
>
> I find I write code like this[*] a fair bit, since my major use for Python is to write remote monitors for embedded kit, so it's pretty much a real world example. I don't find the first version using ":=" to be perfectly clear, in fact I think it's rather ugly. That may be partly the same reaction that many of us had to the asymmetry of assignment expressions in (over-)complicated comprehensions. The second version using "given" reads much more naturally to the mathematician in me, and not too badly to my English half either.

I would write this using a for loop and the two-argument form of iter:

for cmd in iter(get_command, ''):
if cmd.token == CMD_QUIT:
break
cmd.do_something()

or

from itertools import take while

for cmd in takewhile(lambda x: x.token != CMD_QUIT, iter(get_command, '')):
cmd.do_something()

Depending on what get_command actually returns, you might be able to construct a valid sentinel that
doesn't require an explicit test of cmd.token.

(This reminds that I wish ``iter`` could take a predicate instead of a sentinel as its second argument. Then
you could just write

for cmd in iter(get_command, lambda x: x.token == CMD_QUIT):
cmd.do_something()
)

--

Clint

[Kirill Balunov]

2018-05-11 17:06 GMT+03:00 Clint Hepner <clint....@gmail.com>:

(This reminds that I wish ``iter`` could take a predicate instead of a sentinel as its second argument. Then
you could just write

    for cmd in iter(get_command, lambda x: x.token == CMD_QUIT):
        cmd.do_something()
)

But you can do it right now:

class P:
    def __init__(self, key):
        self.key = key
    def __eq__(self, other):
        return self.key(other)

for cmd in iter(get_command, P(lambda x: x.token == CMD_QUIT)):
    cmd.do_something()

With kind regards,

-gdg

[Steven D'Aprano]

On Thu, May 10, 2018 at 10:31:00PM -0400, Nick Malaguti wrote:

> One of my hurdles for ":=" is understanding when I should use it rather
> than "=". Should I use it everywhere? Should I use it only where I can't
> use regular "="? Is it a personal choice? Will it become so common that
> I need to think harder because some people will use it really frequently
> or intermix them?

When should I use "x = x + 1" and when should I use "x += 1"? When
should I write alist.sort() and when sorted(alist)? When should I use a
list comp and when should I use a for-loop? I could give a dozen more
examples.

We always have a choice in writing code. There is almost never *only*
one way to do it. Nevertheless, we manage, and I believe that most of
the things which perplex us today will be so trivially easy to
understand tomorrow that we'll wonder what the fuss was all about.

It is sometimes humbling to remember back at the things that we swore
were terrible, terrible mistakes. I hated augmented assignment, until I
gave in and started using them. I hated != instead of <> and I actually
seriously considered using "from __future__ import barry_as_FLUFL" for a
while.

Depending on whether you are a "glass half full" or "glass half empty"
kind of guy, you could say either that:

- we're irrationally risk adverse, and would rather miss out on
something fantastic than risk something slightly not so good;

- or like Stockholm Syndrome victims, we can get used to, and even
learn to enjoy, the most awful crap if it goes on long enough.

(Or possibly both at the same time.)

Personally, I think that in hindsight my dislike of != was irrational
and rather silly, rather than a prophetic realisation that the dropping
of <> began the ruination of Python. YMMV.

> I don't want to see "=" vs ":=" become like semicolons in JavaScript.

The difference between = and := is nothing like automatic semicolon
insertion in Javascript. The Python interpreter is never going to (only
sometimes) insert a colon to make your "name = expression" work and it
is never going to turn two statements into a single broken expression
because you used = instead of := or vice versa.

Of course people can still screw up the precedence and get the wrong
results, but the same applies to all operators and the same solution
applies: when in doubt, add parentheses to make it clear.

Some arguments against := are better than others: will it encourage
people to write unreadable one-liners instead of using multiple
statements? Maybe, but I think most people will show more sense and
restraint. You haven't seen many quadruply-nested list comprehensions,
or ternary if operators nested ten deep, have you? I haven't.

> When I work on a different codebase, am I going to have to follow an
> "always" or "never" for binding expressions?

Yes, of course you are. Some places will be conservative and say Never,
and some will be gung ho and say Always, but the majority will say "Use
them when they make the code better".

Just like when you work on some code bases you will have to always
follow PEP 8, and when you work on other code bases, you will have to
follow different rules.

> Maybe this is all overblown
> and PEP8 direction will keep everyone on the same page, but I guess I
> worry about there being 2 very similar, but not the same, ways to do it.
> What I really like about "given" is it makes it a lot clearer when I
> should use it.

You seem to have two contradictory opinions here. Paraphrasing:

"I worry that the same people who never abuse ternary if by
writing obfuscated one-liners will suddenly turn around and
abuse := binding expressions by writing obfuscated one-liners."

and

"clearly nobody will abuse 'given' binding expressions by
writing obfuscated one-liners, because they don't abuse
ternary if to write obfuscated one-liners."

I understand being pessimistic about the common-sense of my fellow
programmers. I understand being optimistic about the common-sense of my
fellow programmers. I don't understand doing both at the same time.

If people will abuse := they will abuse "given". If they won't abuse
"given", they surely won't abuse := either. Since we have over a quarter
of a century of experience showing that the Python community, as a
whole, tends not to abuse syntax to write unreadable one-liners, I think
that fears about people abusing := are overblown.



--
Steve

[Steven D'Aprano]

On Fri, May 11, 2018 at 11:40:51AM +0200, Jacco van Dorp wrote:

> I dont really like "given".
>
> If we compare:
>
> if m given m = re.match(stuff):
>
> to
>
> if m := re.match(stuff)
>
> then I count 4+(name_length) more tokens and 2 more spaces. Since I
> believe := is perfectly clear, I don't see the reason for a far more
> verbose syntax.

I agree with Jacco here. We have to name the variable twice, even if it
is only used once, and we have a relatively long keyword, five
characters, longer than average for all keywords, and only one char
short of the maximum.

I know the aim isn't to absolutely minimise the number of keystrokes,
but it does seem strange to use such a long symbol which requires
duplicating the target, unless the intent is to discourage people from
using it.


--
Steve

[Chris Angelico]

On Sat, May 12, 2018 at 12:04 AM, Jacco van Dorp <j.van...@deonet.nl> wrote:
>> [João]
>> How do you read something like " while (cmd := get_command()).token != CMD_QUIT:" in plain english?
>
> while open-paren cee em dee colon is call get-underscore-command
> close-paren dot token doesn't equal all-caps cee em dee underscore
> quit colon.
>
> Might be some dutch in there.

While cee-em-dee (from get-command) dot token isn't CMD_Quit

ChrisA

[Steven D'Aprano]

On Fri, May 11, 2018 at 12:37:43PM +0100, Rhodri James wrote:

> Consider:
>
> while (cmd := get_command()).token != CMD_QUIT:
> cmd.do_something()
>
> vs:
>
> while cmd.token != CMD_QUIT given cmd = get_command():
> cmd.do_something()

Okay, considered. I think the first is preferable.

Much earlier in the PEP 572 discussion, I strongly argued in favour
of the

expr as name

syntax on the basis that the most important part of the overall
expression is "expr", not the assignment target, and therefore that
should come first. Even though I have accepted that "as" is not viable,
I still believe that it is preferable to have the expression first, or
if not first, at least as close to the left as we can get it.

This "given" syntax puts the expr part all the way to the far right of
the line. A line which is made all the longer for needing to use "given"
and redundantly state the target name.

It's like we're trying to maximize the distance the eye has to travel
back and forth when reading.

I have to read to the end of the line before I have any idea where cmd
has come from or what it is. The fact that it comes from a "given"
expression comes as a surprise at the end of the line.

Now obviously this doesn't matter if I'm reading lines of code in
careful detail, but I don't do that all the time. I skim code far more
than I read it in careful detail, and the closer things are to the left,
the more likely I am to see them while skimming. The further out they
are, the easier they are to miss.

I think that "given" will *literally* make reading harder, in that the
eye has to travel further to spot the relevant expression while skimming
over code. As I said, I don't think it makes any difference when reading
closely in detail. But most of my reading of code is skimming to find
the relevant line or section, and then read closely. I would probably
skim a hundred lines for every one I read closely.

We read more code than we write, but writing is important too. I think
the verbosity of "given" (six chars versus two) and the redundancy of
needing to repeat the name of the target even if you only use it once
will soon make using this syntax seem like a chore.



--
Steve

[Guido van Rossum]

Maybe you could propose some kind of syntax using "whereas"? (It can be used as a preamble.)

On Fri, May 11, 2018 at 3:05 AM, Greg Ewing <greg....@canterbury.ac.nz> wrote:

Guido van Rossum wrote:

I'd need well-reasoned explanations

My reasoning is essentially the same as what I've already
said about "where". To summarise, "given" sounds like
something an English-speaking mathematician would write,
whereas ":=" doesn't even have an obvious pronunciation.
Some variation on "given" just seems greatly more pythonic
to me.

--
Greg

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--

--Guido van Rossum (python.org/~guido)

[Guido van Rossum]

On Fri, May 11, 2018 at 3:33 AM, Greg Ewing <greg....@canterbury.ac.nz> wrote:

Most people's short-term memory is good enough to remember
that "m" refers to the match object while they read the
next couple of lines. IMO, using a longer name would serve
no purpose and would just clutter things up.

 

Indeed.

A thought just occurred to me. Maybe we need to instigate a cultural shift where people think about style guides as less dictated by hard-coded rules that were "passed down from the mountain" and more as derived from research that we can all understand about usability. A lot more is known about how human perception and various types of memory and learning work than it was when the "7 things plus/minus 2" rule was invented (https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two).

It would be fascinating to imagine a future where language designers could talk about such topic with as much confidence as they talk about the efficiency of hash tables.

[Steven D'Aprano]

On Fri, May 11, 2018 at 03:47:05PM +0200, João Santos wrote:

> How do you read something like " while (cmd := get_command()).token !=
> CMD_QUIT:" in plain english?

I wouldn't if I could avoid it. I hardly ever program by talking about
code in plain English. Often the lines are gobblydegook:

zreplace = '%c%02d%02d' % (sign, h, m) # datetime.py

and even when they are technically pronouncable English:

# subprocess.py
(p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) =
self._get_handles(stdin, stdout, stderr)

my brain would glaze over by the second "p2c". I prefer to read and
write code than speak it, and if I need to discuss it, I prefer to use a
whiteboard so I can write things down.

But if I really needed to, I'd probably start by reading it as:

while open bracket command defined as get-command close bracket dot
token is not equal to command-quit

and then I'd probably drop the "defined" and swap the order around.

Actually, that's not true. I probably wouldn't say that, not in a
real conversation. What I'd probably say is,

So, like, I've got this command object, see, which I get
from calling get-command, right, and so I get the, um,
token attribute, okay, and if that's not equal to the
quit value, I loop until it is. Right?


(And this is why I prefer *writing* code than *saying* code.)


--
Steve

[Tim Peters]

[Gustavo Carneiro]

>>> IMHO, all these toy examples don't translate well to the real world
>>> because they tend to use very short variable names while in real world [good
>>> written code] tends to select longer more descriptive variable names.

[Greg Ewing]

>> I don't believe that's always true. It depends on the context.
>> Sometimes, using long variable names can make code *harder*
>> to read.
>>
>> I don't think there's anything unrealistic about this
>> example:
>>
>> if m given m = pattern.match(the_string):
>> nugget = m.group(2)
>>
>> Most people's short-term memory is good enough to remember
>> that "m" refers to the match object while they read the
>> next couple of lines. IMO, using a longer name would serve
>> no purpose and would just clutter things up.

[Nick Coghlan]

> I've been thinking about this problem, and I think for the If/elif/while
> cases it's actually possible to allow the "binding is the same as the
> condition" case to be simplified to:
>
> if command = pattern.match(the_string):
> ...
> elif command = other_pattern.match(the_string):
> ...
>
> while data = read_data():

Unless there's some weird font problem on my machine, that looks like
a single "equals sign". In which case we'd be reproducing C's
miserable confusion about whether:

if (i = 1)

was a too-hastily-typed spelling of the intended:

if (i == 1)

or whether they were thinking "equals" and typed "=" by mistake.

If so, that would get an instant -1 from any number of core devs, who
have vivid painful memories of being burned by that in C. That's not
just speculation - it came up a number of times in the PEP 572
threads.

> Allowing this would be part of the definition of the if/elif/while statement
> headers, rather than a general purpose assignment expression.
>
> The restriction of the LHS to a simple name target would need to be in the
> AST generator rather than in the grammar, but it's hardly the only case
> where we do that kind of thing. Switching to the given expression form would
> then only be necessary in cases where the condition *wasn't* the same as the
> binding target.
>
> A similar enhancement could be made to conditional expressions (adjusting
> their grammar to permit "EXPR if NAME = EXPR else EXPR") and filter clauses
> in comprehensions (allowing "EXPR for TARGET in EXPR if NAME = EXPR"). In
> essence, "if", "elif", and "while" would all allow for an "implied given"
> clause in order to simplify the 90% case where the desired condition and the
> bound expression are the same.

Spell it ":=" (colon equals) instead, and a few core devs would stop
objecting ;-)

[Steven D'Aprano]

On Fri, May 11, 2018 at 12:33:10PM -0400, Guido van Rossum wrote:

> A thought just occurred to me. Maybe we need to instigate a cultural shift
> where people think about style guides as less dictated by hard-coded rules
> that were "passed down from the mountain" and more as derived from research
> that we can all understand about usability.

That would be fantastic, but it's a real uphill battle.

How long have we known about the effects of low-contrast on readability?

https://www.wired.com/2016/10/how-the-web-became-unreadable/

http://contrastrebellion.com/

And don't get me started on the awful, awful choices made for chart
design and data visualisation. We know how to make good charts:

https://en.wikipedia.org/wiki/Edward_Tufte

and yet:

https://i.redd.it/0w0y1r1ba8x01.jpg



--
Steve

[Rhodri James]

On 11/05/18 15:04, Jacco van Dorp wrote:
> A while ago, we had this gem:
>
> 2018-04-06 8:19 GMT+02:00 Serhiy Storchaka<stor...@gmail.com>:
>> Using currently supported syntax:
>>
>> smooth_signal = [average for average in [0] for x in signal for average in [(1-decay)*average + decay*x]]
> Go ahead and understand that line in 1 go. It's currently legal syntax
> for a running average for a smoothing signal, which remembers
> something about it. (Subject: Proposal: A Reduce-Map Comprehension and
> a "last" builtin)
>
> You're not allowed to work it out bit by bit, just understand the
> entire line or nothing. Any failure of yours proves my point.

Personally I thought it proved the point that you shouldn't be trying to
squash things like that into a list comprehension in the first place,
because

average = 0
smooth_signal = []
for x in signal:
average = (1 - decay) * average + decay * x
smooth_signal.append(average)

is quite a bit more comprehensible.

--
Rhodri James *-* Kynesim Ltd

[Alexander Belopolsky]

On Fri, May 11, 2018 at 11:43 AM Steven D'Aprano <st...@pearwood.info>
wrote:
> ...

> I agree with Jacco here. We have to name the variable twice, even if it
> is only used once, and we have a relatively long keyword, five
> characters, longer than average for all keywords, and only one char
> short of the maximum.

To be fair when counting the keystrokes, you should take into account that
the colon and the parentheses that appear in the := syntax are the upper
register keys that counting the shift require two keystrokes each.

[Brendan Barnwell]

On 2018-05-11 09:17, Steven D'Aprano wrote:
> Much earlier in the PEP 572 discussion, I strongly argued in favour
> of the
>
> expr as name
>
> syntax on the basis that the most important part of the overall
> expression is "expr", not the assignment target, and therefore that
> should come first. Even though I have accepted that "as" is not viable,
> I still believe that it is preferable to have the expression first, or
> if not first, at least as close to the left as we can get it.
>
> This "given" syntax puts the expr part all the way to the far right of
> the line. A line which is made all the longer for needing to use "given"
> and redundantly state the target name.
>
> It's like we're trying to maximize the distance the eye has to travel
> back and forth when reading.
>
> I have to read to the end of the line before I have any idea where cmd
> has come from or what it is. The fact that it comes from a "given"
> expression comes as a surprise at the end of the line.
>
> Now obviously this doesn't matter if I'm reading lines of code in
> careful detail, but I don't do that all the time. I skim code far more
> than I read it in careful detail, and the closer things are to the left,
> the more likely I am to see them while skimming. The further out they
> are, the easier they are to miss.
>

> I think that "given" will*literally* make reading harder, in that the

> eye has to travel further to spot the relevant expression while skimming
> over code. As I said, I don't think it makes any difference when reading
> closely in detail. But most of my reading of code is skimming to find
> the relevant line or section, and then read closely. I would probably
> skim a hundred lines for every one I read closely.

That is an interesting argument --- interesting to me because I agree
with a lot of it, but it leads me to the opposite conclusion, and also
because it highlights some of the relevant factors for me.

The main part I disagree with is that the most important thing is the
definition of expr. Rather, what I think is most important is the role
of expr within the surrounding expression. For simple cases, it doesn't
much matter which comes first:

if x := do_some_stuff():

if x where x = do_some_stuff():

. . . and it's true the latter is a bit more verbose in that case for
little extra benefit. But when the locally-defined value is used within
a more complicated expression (like the quadratic formula example), I
think readability goes down significantly. To appease Tim, instead of
using the quadratic formula, though, I will use a more realistic example
that comes up fairly often for me: wanting to do some kind of
normalization on a piece of data for a comparison, while keeping the
unnormalized data for use within the block:

if some_condition and (stuff:=
get_user_input()).lower().strip().replace('-', ''):

versus

if some_condition and stuff.lower().strip().replace('-', '') given
stuff = get_user_input():

Now, the latter is still more verbose. But to me it is now more
readable, because the assignment does not disrupt the flow of reading
the surrounding expression. This benefit increases the more complicated
the surrounding expression is.

Your point about reading ease is well taken, but also relevant to me is
that we only read a piece of code *for the first time* once. The
advantage of the given-style assignment is that on multiple readings, it
foregrounds how the assigned value is USED, not how it is DEFINED. This
encourages a "top-down" understanding of the expression in which you
first understand the overall picture, and then later "drill down" into
definition of what the components are.

I wonder if some of the disagreement about the relative merits of the
two cases comes from people focusing on different kinds of examples. As
I said, I think the advantages of "cleft assignment" (i.e., where the
assignment is shunted to the end of the line) become more pronounced as
the surrounding expression becomes more complex. They also become
somewhat greater as the definition of the expression becomes more
complex, because there is more to skip over when finding out how the
value is used. But many of the examples we're seeing are very simple
ones, and in particular have a trivial surrounding expression. (That
is, in something like "m := re.match()" the assigned value is not being
used in any larger expression; the assigned value constitutes the whole
of the expression.)

I'm also finding it useful to think of parallel situations in prose
writing, particularly journalistic-style prose writing. The current
Python behavior, in which assignments must be done before the block, I
think of as akin to something like this:

"The IAAEA (International Association of Assignment Expression
Advocates) is an organization dedicated to the promotion of assignment
expressions. Its president is Eddie McEquals. In a statement
yesterday, McEquals called for a new syntax to bring assignment
expressions to the masses."

The inline-assignment syntax is akin to this:

In a statement yesterday, Eddie McEquals, president of the International
Association of Assignment Expression Advocates (IAAEA), an organization
dedicated to the promotion of assignment expressions, called for a new
syntax to bring assignment expressions to the masses.

The cleft-assignment syntax is akin to this:

In a statement yesterday, Eddie McEquals called for a new syntax to
bring assignment expressions to the masses. McEquals is the president
of the International Association of Assignment Expression Advocates
(IAAEA), which is an organization dedicated to the promotion of
assignment expressions.

Now of course I'm fudging a bit on the details (like whether you have
IAAEA in parentheses or its full expansion), but the point is that the
last version foregrounds the "bottom line" or "predicate" --- what
actually happened. The first one foregrounds the "participants", or
who/what was involved, and saves what actually happened for the end.
But the middle one, to my eye, foregrounds nothing. It stuffs
everything into one big clause where the descriptions of the
participants occur as asides that disrupt the flow of reading. By the
time we get to the predicate, we have to jump back to the beginning to
remember who it is we're talking about,

Again, the difference in readability varies depending on the complexity
of the different components. If we just have "Eddie McEquals, president
of the International Association of Assignment Expression advocates,
delivered a speech yesterday at the organization's convention", the
inline appositive is not so disruptive. But the more complex the inline
definitions become, and especially the more complex the expression in
which they are embedded, the lower readability goes, for me.

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."
--author unknown

[Brendan Barnwell]

On 2018-05-11 11:08, Brendan Barnwell wrote:
> . . . and it's true the latter is a bit more verbose in that case for
> little extra benefit. But when the locally-defined value is used within
> a more complicated expression (like the quadratic formula example), I
> think readability goes down significantly. To appease Tim, instead of
> using the quadratic formula, though, I will use a more realistic example
> that comes up fairly often for me: wanting to do some kind of
> normalization on a piece of data for a comparison, while keeping the
> unnormalized data for use within the block:
>
> if some_condition and (stuff:=
> get_user_input()).lower().strip().replace('-', ''):
>
> versus
>
> if some_condition and stuff.lower().strip().replace('-', '') given
> stuff = get_user_input():

Ironically I weakened my argument by forgetting to finish my expression
there. I intended that chain of method calls to be used in a comparison
to make the surrounding expression more complex. So revise the above to

if some_condition and (stuff :=

get_user_input()).lower().strip().replace('-', '') == existing_value:

versus

if some_condition and stuff.lower().strip().replace('-', '') ==
existing_value given stuff = get_user_input():

[Tim Peters]

[Brendan Barnwell]

>>
>> . . . and it's true the latter is a bit more verbose in that case for
>> little extra benefit. But when the locally-defined value is used within
>> a more complicated expression (like the quadratic formula example), I
>> think readability goes down significantly. To appease Tim, instead of
>> using the quadratic formula, though, I will use a more realistic example
>> that comes up fairly often for me: wanting to do some kind of
>> normalization on a piece of data for a comparison, while keeping the
>> unnormalized data for use within the block:
>>
>> if some_condition and (stuff:=
>> get_user_input()).lower().strip().replace('-', ''):
>>
>> versus
>>
>> if some_condition and stuff.lower().strip().replace('-', '') given
>> stuff = get_user_input():

[also Brendan]

> Ironically I weakened my argument by forgetting to finish my
> expression there. I intended that chain of method calls to be used in a
> comparison to make the surrounding expression more complex. So revise the
> above to
>
> if some_condition and (stuff :=
> get_user_input()).lower().strip().replace('-', '') == existing_value:
>
> versus
>
> if some_condition and stuff.lower().strip().replace('-', '') ==
> existing_value given stuff = get_user_input():

Even more ironically, to my eyes the original more strongly supported
your view than the rewrite ;-)

"given stuff =" stuck out in the original because it was preceded by
punctuation (a right parenthesis).

I had to read the rewrite 3 times before i realized you were even
_using_ "given", because there it's buried between two other names -
"existing_value given stuff" - and visually looks more like it's
actually the 3rd of 4 words (because of the underscore in
"existing_value").

Of course that would have been obvious in a Python-aware editor that
colored "given" differently, but as-is I found the original easy to
read but the rewrite a puzzle to decode.

Similarly, in the rewritten assignment expression spelling, it's
obvious at a glance that the test is of the form

some_condition and some_messy_expression == existing_value

but in the rewritten "given" sample that's obscured because
"existing_value" not only doesn't end the statement, it's not even
followed by punctuation. Of course coloring "given" differently would
remove that visual uncertainty too. For a dumb display, I'd write it

if some_condition and (
stuff.lower().strip().replace('-', '') == existing_value) given
stuff = get_user_input():

instead (added parens so that "existing_value" and "given" are
separated by punctuation).

[Matt Arcidy]

Apology for top post, but this is a general statement about Readability and not a response to an individual.

it would be nice to list the objective parts separate from the "argument" (i.e. debate, not fight), perhaps list them then make a case for which metric is a more important, and which values produce better results.

Key strokes, keyboard location, location of information vs where it is used, cognitive load (objective barring neuroscience changes) are all objective points (along with many other points raised).

"Better" will be decided by Guido I guess, but listing objective points with explanatory examples gives a basis for that discussion.

Legibility, for example, is not objective at all, it has nothing to do with syntax.  This covers fonts, colors, monitors, lighting, chairs, lunch, etc.  None of this is relevent to the symbols or their ordering in a file we all must read.

Teachability likewise.  My opinion here is learnability is far more important anyways, I am 90% self taught going back 25 years, but this is equally unquantifiable.  Perhaps just trust students to learn as an author must trust a reader.

Of course, let it not be lost that determining teachability and learnability for something which doesn't even exist yet is quite challenging.  

Any quantification will give more information than only naked impassioned pleas to Readability.  

Note Tim came up with a real metric:

2 * count(":=")/len(statement).

It's objective.  it's just unclear if a higher score is better or worse.  However, one could say "a Tim of .3 is considered too high" as a guideline. 

Perhaps coders find these opportunities to express feelings and opinion cathartic after speaking to the most pedantic creature on Earth (compilier/computer) but I think exercising the high skill level available here to dissect and find objective statements is a worthy puzzle.

[Ryan Gonzalez]

There are some variants of tanks like 'if let' where the bindings come
*first*, unlike 'given' where they come last (like Haskell's 'where').

>
> Of course that would have been obvious in a Python-aware editor that
> colored "given" differently, but as-is I found the original easy to
> read but the rewrite a puzzle to decode.

Well, you've partly explained the reason: our eyes are drawn to what sticks
out. In this case, the := stuck out in a section heavy on black letters. In
a proper editor, it may even be the other way around: they tend to
highlight keywords in a stronger manner (like bolding) than operators.

[Angus Hollands]

I'd like to address Steven D'Aprano's reply earlier in the list concerning "given" vs ":=" clauses.

My stance on the PEP is that the general ability to assign locally within an expression is undesirable. I gave several reasons, but in general, it means processing lines becomes more complex, and it makes refactoring more difficult. I don't think that there are enough cases where it would be useful to justify its negatives. I can recall but several occasions (writing regexes, mainly) that I've wanted something like this.

However, if we were to assert that PEP 572 was mandatory, and then dispute its implementation, then I would argue as follows:

Cost:
A lot of people seem quite fixed upon the character cost of "given" vs ":=". I think this is a straw man. Most people spend longer reading than writing code, so if you agree with that observation, then it's the line length that you are concerned with. While it is true that overly long lines are not desirable, it's also true that incredibly short lines are equally difficult to understand. I don't think that keyword length can be independently taken too seriously as support for any proposal. Once you have to include grouping parentheses, this length difference quickly diminishes anyway.

Repetition of variable:
In addition, several people seem unhappy that the "given" keyword approach sees a repetition of the state variable in question. I think this is actually a benefit for two reasons. Firstly, you can save the expression in question to the intermediate variable, and then access a sub expression in a condition

e.g

while value.in_use given value = get_next_pool_item():
    print(value.refcount())

Instead of 

while (value:=get_next_pool_item()) and value.in_use:
    print(value.refcount())

Secondly, it reads in the order that matters. When reading the first line, one encounters what the condition is evaluating first, and then the implementation details (m=p.match) second. It reads as one would describe a mathematical equation in a paper, and clearly separates what you're interested in from what it depends upon. This is what I particularly dislike about the ":=" operator approach, the value, and name it is bound to, are unrelated at the point of evaluation, yet are right next to each other. It's visual noise. In the example above, the reader has to read the entire line when trying to find the loop condition. 

Someone earlier suggested this was a bad thing, that the expression was far to the right of the line. I disagree. In cases where you might want to assign an expression to a variable, it is going to be used at least twice (otherwise just use the expression directly). At this point, the target name should be explicit in what it represents. This depends upon context slightly, such as what is happening in the local region of code. An example; when matching regexes, the match conditions (if x.group(2) == ...) are more important than what you matched on, usually. 

In addition, the two cases are not identical - if the API is such that get_next_pool_item should not return None values, this wouldn't be caught in the ":=" approach, unless you add more conditions. Yes, you could argue that I've cherry picked an example here. Actually, I haven't; I observed this after writing the example. 

What am I getting at here? In effect, the "given" keyword provides a superset of use cases to that of ":=". Dare I say it, but explicit is better than implicit.

Readability:
A smaller point is that I don't feel that ":=" is very readable. If we had to use an operator, I think $= is better, but me reasoning for this is weak. I think it derives from my observation that ":=" is slow to distinguish from "=".

Regards,
Angus Hollands

On Fri, 11 May 2018 at 17:45 <python-ide...@python.org> wrote:

Send Python-ideas mailing list submissions to
        python...@python.org

To subscribe or unsubscribe via the World Wide Web, visit
        https://mail.python.org/mailman/listinfo/python-ideas
or, via email, send a message with subject or body 'help' to
        python-ide...@python.org

You can reach the person managing the list at
        python-id...@python.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Python-ideas digest..."


Today's Topics:

   1. Re: Inline assignments using "given" clauses (Steven D'Aprano)
   2. Re: Inline assignments using "given" clauses (Guido van Rossum)
   3. Re: Inline assignments using "given" clauses (Guido van Rossum)
   4. Re: Inline assignments using "given" clauses (Steven D'Aprano)
   5. Re: Inline assignments using "given" clauses (Tim Peters)


----------------------------------------------------------------------

Message: 1
Date: Sat, 12 May 2018 02:17:24 +1000
From: Steven D'Aprano <st...@pearwood.info>
To: python...@python.org
Subject: Re: [Python-ideas] Inline assignments using "given" clauses
Message-ID: <2018051116...@ando.pearwood.info>
Content-Type: text/plain; charset=us-ascii

On Fri, May 11, 2018 at 12:37:43PM +0100, Rhodri James wrote:

> Consider:
>

>   while (cmd := get_command()).token != CMD_QUIT:

>     cmd.do_something()
>
> vs:
>
>   while cmd.token != CMD_QUIT given cmd = get_command():
>     cmd.do_something()

Okay, considered. I think the first is preferable.

Much earlier in the PEP 572 discussion, I strongly argued in favour
of the

    expr as name

syntax on the basis that the most important part of the overall
expression is "expr", not the assignment target, and therefore that
should come first. Even though I have accepted that "as" is not viable,
I still believe that it is preferable to have the expression first, or
if not first, at least as close to the left as we can get it.

This "given" syntax puts the expr part all the way to the far right of
the line. A line which is made all the longer for needing to use "given"
and redundantly state the target name.

It's like we're trying to maximize the distance the eye has to travel
back and forth when reading.

I have to read to the end of the line before I have any idea where cmd
has come from or what it is. The fact that it comes from a "given"
expression comes as a surprise at the end of the line.

Now obviously this doesn't matter if I'm reading lines of code in
careful detail, but I don't do that all the time. I skim code far more
than I read it in careful detail, and the closer things are to the left,
the more likely I am to see them while skimming. The further out they
are, the easier they are to miss.

I think that "given" will *literally* make reading harder, in that the

eye has to travel further to spot the relevant expression while skimming
over code. As I said, I don't think it makes any difference when reading
closely in detail. But most of my reading of code is skimming to find
the relevant line or section, and then read closely. I would probably
skim a hundred lines for every one I read closely.

We read more code than we write, but writing is important too. I think
the verbosity of "given" (six chars versus two) and the redundancy of
needing to repeat the name of the target even if you only use it once
will soon make using this syntax seem like a chore.



--
Steve

------------------------------

Message: 2
Date: Fri, 11 May 2018 12:25:46 -0400
From: Guido van Rossum <gu...@python.org>
To: Greg Ewing <greg....@canterbury.ac.nz>
Cc: python-ideas <python...@python.org>
Subject: Re: [Python-ideas] Inline assignments using "given" clauses
Message-ID:
        <CAP7+vJLR9N4N1g4U3T-DgYGMHAM_a=inDi7dFiF7...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Maybe you could propose some kind of syntax using "whereas"? (It can be
used as a preamble.)

On Fri, May 11, 2018 at 3:05 AM, Greg Ewing <greg....@canterbury.ac.nz>
wrote:

> Guido van Rossum wrote:
>
>> I'd need well-reasoned explanations
>>
>
> My reasoning is essentially the same as what I've already
> said about "where". To summarise, "given" sounds like
> something an English-speaking mathematician would write,
> whereas ":=" doesn't even have an obvious pronunciation.
> Some variation on "given" just seems greatly more pythonic
> to me.
>
> --
> Greg
>

> _______________________________________________
> Python-ideas mailing list
> Python...@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>

--
--Guido van Rossum (python.org/~guido)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180511/d6443de1/attachment-0001.html>

------------------------------

Message: 3
Date: Fri, 11 May 2018 12:33:10 -0400
From: Guido van Rossum <gu...@python.org>
To: Greg Ewing <greg....@canterbury.ac.nz>
Cc: Python-Ideas <python...@python.org>
Subject: Re: [Python-ideas] Inline assignments using "given" clauses
Message-ID:
        <CAP7+vJJ-yEyOnO8N4kmyyxs3qAjFyPFv=7SU3GT3T...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

On Fri, May 11, 2018 at 3:33 AM, Greg Ewing <greg....@canterbury.ac.nz>
wrote:

> Most people's short-term memory is good enough to remember
> that "m" refers to the match object while they read the
> next couple of lines. IMO, using a longer name would serve
> no purpose and would just clutter things up.

Indeed.

A thought just occurred to me. Maybe we need to instigate a cultural shift
where people think about style guides as less dictated by hard-coded rules
that were "passed down from the mountain" and more as derived from research

that we can all understand about usability. A lot more is known about how
human perception and various types of memory and learning work than it was
when the "7 things plus/minus 2" rule was invented (
https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two).

It would be fascinating to imagine a future where language designers could
talk about such topic with as much confidence as they talk about the
efficiency of hash tables.

--
--Guido van Rossum (python.org/~guido)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180511/808769ad/attachment-0001.html>

------------------------------

Message: 4
Date: Sat, 12 May 2018 02:43:14 +1000
From: Steven D'Aprano <st...@pearwood.info>
To: python...@python.org
Subject: Re: [Python-ideas] Inline assignments using "given" clauses
Message-ID: <2018051116...@ando.pearwood.info>
Content-Type: text/plain; charset=iso-8859-1

On Fri, May 11, 2018 at 03:47:05PM +0200, Jo?o Santos wrote:

> How do you read something like " while (cmd := get_command()).token !=
> CMD_QUIT:" in plain english?

I wouldn't if I could avoid it. I hardly ever program by talking about
code in plain English. Often the lines are gobblydegook:

    zreplace = '%c%02d%02d' % (sign, h, m)  # datetime.py

and even when they are technically pronouncable English:

    # subprocess.py   
    (p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) =
            self._get_handles(stdin, stdout, stderr)

my brain would glaze over by the second "p2c". I prefer to read and
write code than speak it, and if I need to discuss it, I prefer to use a
whiteboard so I can write things down.

But if I really needed to, I'd probably start by reading it as:

    while open bracket command defined as get-command close bracket dot
    token is not equal to command-quit

and then I'd probably drop the "defined" and swap the order around.

Actually, that's not true. I probably wouldn't say that, not in a
real conversation. What I'd probably say is,

    So, like, I've got this command object, see, which I get
    from calling get-command, right, and so I get the, um,
    token attribute, okay, and if that's not equal to the
    quit value, I loop until it is. Right?


(And this is why I prefer *writing* code than *saying* code.)


--
Steve

------------------------------

Message: 5
Date: Fri, 11 May 2018 11:45:13 -0500
From: Tim Peters <tim.p...@gmail.com>
To: Nick Coghlan <ncog...@gmail.com>
Cc: Greg Ewing <greg....@canterbury.ac.nz>, Python-Ideas
        <python...@python.org>
Subject: Re: [Python-ideas] Inline assignments using "given" clauses
Message-ID:
        <CAExdVNnm0AG4KyjWg-w8CHkoVMh4iPbJtFzO4eApTKLK23=q...@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"

------------------------------

Subject: Digest Footer

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas

------------------------------

End of Python-ideas Digest, Vol 138, Issue 90
*********************************************

[Chris Angelico]

On Sat, May 12, 2018 at 7:12 AM, Angus Hollands <goos...@gmail.com> wrote:
> e.g
>
> while value.in_use given value = get_next_pool_item():
> print(value.refcount())
>
> Instead of
>
> while (value:=get_next_pool_item()) and value.in_use:
> print(value.refcount())
>

> In addition, the two cases are not identical - if the API is such that
> get_next_pool_item should not return None values, this wouldn't be caught in
> the ":=" approach, unless you add more conditions. Yes, you could argue that
> I've cherry picked an example here. Actually, I haven't; I observed this
> after writing the example.
>
> What am I getting at here? In effect, the "given" keyword provides a
> superset of use cases to that of ":=". Dare I say it, but explicit is better
> than implicit.

I'm not sure what you're getting at here. To make your two versions
equivalent, you'd need to write them like this:

while value and value.in_use given value = get_next_pool_item():
print(value.refcount())

while (value:=get_next_pool_item()) and value.in_use:
print(value.refcount())

or like this:

while value.in_use given value = get_next_pool_item():
print(value.refcount())

while (value:=get_next_pool_item()).in_use:
print(value.refcount())

There, now they're identical. There's no superset or subset of use
cases. And I have no idea how that connects with the oft-misused
"explicit is better than implicit".

(I'm still fairly sure that "explicit" and "strongly typed" are both
synonyms for "stuff I like", with their antonyms "implicit" and
"weakly typed" both being synonyms for "stuff I don't like". Years of
discussion have not disproven this theory yet.)

ChrisA

[Ed Kellett]

On 2018-05-11 22:12, Angus Hollands wrote:
> while (value:=get_next_pool_item()) and value.in_use:
> print(value.refcount())

Just as a heads-up, I believe the prescribed way of doing that is:

while (value := get_next_pool_item()).in_use:

Of course you'd need additional mess to do something else with value. I
don't like the asymmetry here:

while (value := get_next_pool_item()).in_use and value is not blah:

> Secondly, it reads in the order that matters. When reading the first line,

> one encounters what the condition is evaluating *first*, and then the

> implementation details (m=p.match) second. It reads as one would describe a

> mathematical equation in a paper, and clearly separates *what you're
> interested in* from *what it depends upon*. This is what I particularly

> dislike about the ":=" operator approach, the value, and name it is bound
> to, are unrelated at the point of evaluation, yet are right next to each
> other. It's visual noise. In the example above, the reader has to read the
> entire line when trying to find the loop condition.

I'm inclined to agree. But several people have argued that this is more
readable than the alternative. I don't buy the reasoning, but they still
like it better, and there's probably no point in going any further into
this aspect. I doubt people are going to be convinced.

> What am I getting at here? In effect, the "given" keyword provides a

> superset of use cases to that of ":=". Dare I say it, but *explicit is
> better than implicit*.

I'm not sure that it's strictly a superset. It's arguably the reverse,
since it's restricted to statements with a condition rather than
arbitrary expressions. I think the more important thing is that
it's--subjectively--better at the subset of use cases that people seem
to actually have (as listed in the OP).

> *Readability:*

> A smaller point is that I don't feel that ":=" is very readable. If we had
> to use an operator, I think $= is better, but me reasoning for this is
> weak. I think it derives from my observation that ":=" is slow to
> distinguish from "=".

Clearly the objectively best choice is "<-".

[Chris Angelico]

On Sat, May 12, 2018 at 7:56 AM, Ed Kellett <e+pytho...@kellett.im> wrote:
>> *Readability:*
>> A smaller point is that I don't feel that ":=" is very readable. If we had
>> to use an operator, I think $= is better, but me reasoning for this is
>> weak. I think it derives from my observation that ":=" is slow to
>> distinguish from "=".
>
> Clearly the objectively best choice is "<-".

It's already legal syntax. That's an advantage, right?

ChrisA

[Steven D'Aprano]

On Fri, May 11, 2018 at 11:52:02AM -0700, Matt Arcidy wrote:

> Note Tim came up with a real metric:
> 2 * count(":=")/len(statement).
> It's objective. it's just unclear if a higher score is better or worse.
> However, one could say "a Tim of .3 is considered too high" as a guideline.

I think Tim was making a joke about demanding objective measurements of
subjective things.

Certainly he hasn't done any research or study to justify that metric.
He just plucked the formula out of thin air.

Or at least no peer reviewed research.


--
Steve

[Steven D'Aprano]

On Fri, May 11, 2018 at 01:42:55PM -0400, Alexander Belopolsky wrote:
> On Fri, May 11, 2018 at 11:43 AM Steven D'Aprano <st...@pearwood.info>
> wrote:
> > ...
> > I agree with Jacco here. We have to name the variable twice, even if it
> > is only used once, and we have a relatively long keyword, five
> > characters, longer than average for all keywords, and only one char
> > short of the maximum.
>
> To be fair when counting the keystrokes, you should take into account that
> the colon and the parentheses that appear in the := syntax are the upper
> register keys that counting the shift require two keystrokes each.

That's a good point, but also to be fair, I didn't mention keystrokes,
only characters :-)


--
Steve

[Steven D'Aprano]

On Sat, May 12, 2018 at 07:44:24AM +1000, Chris Angelico wrote:

> (I'm still fairly sure that "explicit" and "strongly typed" are both
> synonyms for "stuff I like", with their antonyms "implicit" and
> "weakly typed" both being synonyms for "stuff I don't like". Years of
> discussion have not disproven this theory yet.)

That's certainly how they are used the great majority of the time.

A bit like how "strawman argument" is mostly used to mean "dammit, you
just spotted an unwelcome consequence and/or flaw in my position which I
have no counter too".

:-)


--
Steve

[Tim Peters]

[Matt Arcidy]

>> Note Tim came up with a real metric:
>> 2 * count(":=")/len(statement).
>> It's objective. it's just unclear if a higher score is better or worse.
>> However, one could say "a Tim of .3 is considered too high" as a guideline.

[Steven D'Aprano]

> I think Tim was making a joke about demanding objective measurements of
> subjective things.
>
> Certainly he hasn't done any research or study to justify that metric.
> He just plucked the formula out of thin air.

It was the outcome of an intense 17-year research project.

> Or at least no peer reviewed research.

Au contraire! My peers are here, and that message was reviewed by at
least 3 people on this list.

That said, I am a fan of objectively measuring subjective things, just
not of taking the measurements seriously ;-)

If people do want to take it seriously, check out prior Python art first:

http://radon.readthedocs.io/en/latest/intro.html

[Juancarlo Añez]

>   while (cmd := get_command()).token != CMD_QUIT:
>     cmd.do_something()

while get_command() as cmd:

    if cmd.token == CMD_QUIT:

        break

    cmd.do_something()

--

Juancarlo Añez

[Eric V. Smith]

On 5/11/18 5:12 PM, Angus Hollands wrote:

> *Readability:*

> A smaller point is that I don't feel that ":=" is very readable. If we
> had to use an operator, I think $= is better, but me reasoning for this
> is weak. I think it derives from my observation that ":=" is slow to
> distinguish from "=".

:= would prevent you from using assignment expressions inside f-strings,
which could be argued is a good thing.

To demonstrate, and just for giggles, this works in 3.6, and appears to
have the desired behavior:

--------------------------
class X:
def __init__(self, value):
self.value = value
def __str__(self):
return str(self.value)
def __format__(self, fmt):
assert fmt[0] == '='
self.value = eval(fmt[1:])
return ''

x = X(3)
print(x)
f'{x:=4}' # Behold!
print(x)
--------------------------

Produces:
3
4

I kid, of course.

Eric

[Rob Cliffe via Python-ideas]

Yeah, well, I'm totally lost.  Even after trying out this code, and
refactoring it once if not twice, I didn't understand it.  I don't know
what point you're trying to prove, but you seem to have comprehensively
proved it.

On 11/05/2018 15:04, Jacco van Dorp wrote:
> A while ago, we had this gem:
>
> 2018-04-06 8:19 GMT+02:00 Serhiy Storchaka <stor...@gmail.com>:
>> Using currently supported syntax:
>>
>> smooth_signal = [average for average in [0] for x in signal for average in [(1-decay)*average + decay*x]]
> Go ahead and understand that line in 1 go. It's currently legal syntax
> for a running average for a smoothing signal, which remembers
> something about it. (Subject: Proposal: A Reduce-Map Comprehension and
> a "last" builtin)
>
> You're not allowed to work it out bit by bit, just understand the
> entire line or nothing. Any failure of yours proves my point.
>

>> [João]
>> How do you read something like " while (cmd := get_command()).token != CMD_QUIT:" in plain english?
> while open-paren cee em dee colon is call get-underscore-command
> close-paren dot token doesn't equal all-caps cee em dee underscore
> quit colon.
>
> Might be some dutch in there.
>

> But far more importantly, I can hold the concept into my head, or just
> the parts of it that I need. How we call it in english is actually not
> a good argument - whether we can easily mentally parse it is, since I
> tend not to code by voice command, but with a keyboard. Your mileage
> may vary, but I think we should optimize for keyboard coding over
> voice chat coding. And when I need to refer to it, I say "this bit
> here" or I copy paste it.

> _______________________________________________
> Python-ideas mailing list
> Python...@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>

> ---
> This email has been checked for viruses by AVG.
> http://www.avg.com

[Steven D'Aprano]

On Sat, May 12, 2018 at 02:37:20AM +0100, Rob Cliffe via Python-ideas wrote:

> Yeah, well, I'm totally lost.  Even after trying out this code, and
> refactoring it once if not twice, I didn't understand it.  I don't know
> what point you're trying to prove, but you seem to have comprehensively
> proved it.

Do you mean Serhiy's example of currently supported syntax?

smooth_signal = [average for average in [0] for x in signal for average in [(1-decay)*average + decay*x]]

It helps if you know the algorithm for exponential smoothing: for
each value x (aside from the first), the average is equal to a mix
of the current value x and the previous average A, split by some
proportion P:

A = (1-P)*A + P*x

If P is 0.5, that is equivalent to taking the ordinary average between
the current value and the previous average:

A = (A+x)/2 # when P == 0.5

In the comprehension, P is called "decay" and A is called "average":

average = (1-decay)*average + decay*x

Writing the comprehension as a single line is hard to read. Let's give
it some structure:

smooth_signal = [average # append average to the results

for average in [0]
for x in signal
for average in [(1-decay)*average + decay*x]
]

Horrible as it is, it is perfectly legal Python right now. It uses

for name in SINGLE_ITEM_LIST

to perform an assignment. So that's equivalent to:

average = 0
for x in signal
average = (1-decay)*average + decay*x
append average to the results


Pull the initial value of average out of the comprehension, and use the
PEP 572 syntax:

average = 0
smooth_signal = [(average := (1-decay)*average + decay*x) for x in signal]

which is a huge improvement in my opinion. It would be more obvious if
the expression being calculated came first:

smooth_signal = [(1-decay)*average + decay*x as average for x in signal]

but there are good reasons why the "as" syntax won't work. So it looks
like we're stuck with needing to look ahead past the := to see the
actual value being appended to the list.

A minor inconvenience, equivalent to that in ternary if, where we have
to look ahead to see the condition:

[target := COMPREHENSION_VALUE for x in sequence]

true_value if CONDITION else false_value


So I expect that it will take me a little while to learn to look ahead
and read binding-expressions fluently. (Like comprehensions themselves,
really. It took me a few months to stop needing to pull them apart to
understand them.)

He's Nick's version, as best as I am able to tell:

average = 0
smooth_signal = [(average given average = (1-decay)*average + decay*x) for x in signal]

So we have the same look-ahead needed to see the expression we care
about, but instead of merely having two characters := needed to do the
binding, we need "given average =".


--
Steve

[Matt Arcidy]

On Fri, May 11, 2018, 17:04 Tim Peters <tim.p...@gmail.com> wrote:

[Matt Arcidy]
>> Note Tim came up with a real metric:
>> 2 * count(":=")/len(statement).
>> It's objective.  it's just unclear if a higher score is better or worse.
>> However, one could say "a Tim of .3 is considered too high" as a guideline.

[Steven D'Aprano]
> I think Tim was making a joke about demanding objective measurements of
> subjective things.
>
> Certainly he hasn't done any research or study to justify that metric.
> He just plucked the formula out of thin air.

It was the outcome of an intense 17-year research project.


> Or at least no peer reviewed research.

Au contraire!  My peers are here, and that message was reviewed by at
least 3 people on this list.

That said, I am a fan of objectively measuring subjective things, just
not of taking the measurements seriously ;-)

apparently my joke was objectively not funny :-)  I thought calling it a "Tim" was sufficient.

Im not serious about actually ranking for the purposes of a PEP.  I brought it up when I felt the subjectivity was making the debate worse.

Reiterating my point, n long sub-threads about fonts, screens, etc are ridiculous when those exist outside the pyfile.  I don't know why personal preference for a font would stop a useful tool.  Hopefully those arguments are ignored.  Likewise for googlability, teachability, cross-language similarity and familiarity.  If the tool is useful, that's all that will matter with respect to these points, they solve themselves.

I happen to be working on a ranking tool for code (not quality, just an ordering to find entry points for new devs), so i tossed the idea in.  it seemed appropriate to remind people that the fact that not everyone uses green to highlight "+" doesn't make "+" somehow more or less _useful_ (people -1'd just for legibility alone because of their personal feelings)

I'm not sure where people stand on usefulness, but it's clear this tool is a pattern.  No counter example of "but I can already do this" is related to other counter examples in the way that named expressions solves all of them, and does it succinctly regardless of chosen syntax.  Some required imports! 

Obviously making these decisions with the future unknown is nearly impossible and requires careful consideration of all points, but I don't think Hypothetical Bill's perscription glasses should determine syntax decisions.

Best of luck with the hard parts, clearly I hope the PEP makes it.

If people do want to take it seriously, check out prior Python art first:

Awesome, thanks!  

[Kirill Balunov]

2018-05-12 11:36 GMT+03:00 Angus Hollands <goos...@gmail.com>:

Concerning my previous email,

Yes, my mistake. I'd forgotten (ironically) that the whole point is that it's an expression itself.

So
> while (value:=get_next_pool_item()).in_use:
>    print(value.refcount())
would be the appropriate analogue.

Consequently, my example is invalid. A better example would have been where one needs to access more than one attribute of the expression

while (node.x, node.y) > (5.0, 5.0) given node = get_neighbours(node):
    pass

In addition, this form gives you all the advantages of tuple unpacking:

while (x, y) > (5.0, 5.0) given x, y = get_neighbours(node):

    pass

There was some criticism about the length of the `given`, maybe it is possible to _repurpose_ `with` keyword:

while (x, y) > (5.0, 5.0) with x, y = get_neighbours(node):

    pass

In this context, the with statement and with expression are clearly distinguishable both for the parser and for the person. But maybe many will find this as a bad style ... since the semantics are too different.

With kind regards,

-gdg

[Neil Girdhar]

I love given compared with := mainly because

Simpler is better than complex:

* given breaks a complex statement into two simpler ones, which is putting people off in the simple examples shown here (some people are annoyed by the extra characters).  However, when given is used in a list comprehension to prevent having to re-express it as a for loop, then two simple statements are easier to understand than one complex statement.   This is a common difference between code written at programming contests versus code written by those same software engineers years later at big companies.  Code that you write for yourself can be compact because you already understand it, but code you write professionally is read many many more times than it is written.  Accessibility is much more important than concision. 

* Python has a reputation for being working pseudocode, and given reads like pseudocode.  := needs to be deciphered by comparison especially in the complicated cases where multiple := operators are used on one line.

* there are no difficult mental questions about evaluation order, e.g., in a bracketed expression having multiple assignments.  Similarly, instead of (a.b(a) given a = c.d())  do I write (a.b(a := c.d())) or ((a := c.d()).b(a)) ?

* it avoids the question of what happens when := is used in a switch:  (a if (b := c) else d)   Sometimes you want the assignment to happen unconditionally (a if (b:=c) else d) + b; sometimes you don't.  How do you force one case or the other?  given makes it obvious by separating assignment from the usage of its assignment target.

Style:

* it avoids the big style question of when to use and when not to use :=.  (Even if you ask people not to, people are going to write the expression-statement a := b as a synonym for the statement a = b.)

* it looks a lot like the existing Python "for" and "if" clauses, which also do in-expression assignments.  This makes formatting the code obvious too:

(expression

 given a = b)

compared with

expresion (

    a := b

) rest of expression

which quickly gets ugly.

Best, Neil

On Thursday, May 10, 2018 at 9:46:01 AM UTC-4, Guido van Rossum wrote:

I'm sorry, but unless there's a sudden landslide of support for 'given' in favor of ':=', I'm really not going to consider it.

I'd pronounce "if (x := y) > 0" as either "if y (assigned to x) is greater than zero" or "if x (assigned from y) is greater than zero".

On Thu, May 10, 2018 at 6:39 AM, Nick Coghlan <ncog...@gmail.com> wrote:

On 8 May 2018 at 04:19, Brett Cannon <br...@python.org> wrote:

My brain wants to drop the variable name in front of 'given':

if given m = pattern.search(data):

while given m = pattern.search(remaining_data):

Maybe it's because the examples use such a short variable name?

Does that change if the condition isn't just "bool(name)"? For example:

    if y > 0 given y = f(x):
        ...

That's the situation where I strongly prefer the postfix operator spelling, since it's pretty clear how I should pronounce it (i.e. "if y is greater than zero, given y is set to f-of-x, then ..."). By contrast, while a variety of  plausible suggestions have been made, I still don't really know how to pronounce "if (y := f(x)) > 0:)" in a way that's going to be clear to an English-speaking listener (aside from pronouncing it the same way as I'd pronounce the version using "given", but that then raises the question of "Why isn't it written the way it is pronounced?").

I do agree with Tim that the name repetition would strongly encourage the use of short names rather than long ones (since you're always typing them at least twice), such that we'd probably see code like:

    while not probable_prime(n) given (n =
                            highbit | randrange(1, highbit, 2)):
        pass

Rather than the more explicit:

    while not probable_prime(candidate) given (candidate =
                            highbit | randrange(1, highbit, 2)):
        pass

However, I'd still consider both of those easier to follow than:

    while not probable_prime(candidate := highbit | randrange(1, highbit, 2)):
        pass

since it's really unclear to me that "candidate" in the latter form is a positional argument being bound to a name in the local environment, and *not* a keyword argument being passed to "probable_prime".

I've also been pondering what the given variant might look like as a generally available postfix operator, rather than being restricted to if/elif/while clauses, and I think that would have interesting implications for the flexibility of its usage in comprehensions, since there would now be *three* places where "given" could appear (as is already the case for the inline binding operator spelling):

- in the result expression

- in the iterable expression

- in the filter expression

That is:

    [(x, y, x - y) given y = f(x) for x in data]

    [(x, data) for x in data given data = get_data()]

    [(x, y, x/y) for x in data if y given y = f(x)]

Rather than:

    [(x, y := f(x), x - y) for x in data]

    [(x, data) for x in data := get_data()]

    [(x, y, x/y) for x in data if y := f(x)]

Opening it up that way would allow for some odd usages that might need to be discouraged in PEP 8 (like explicitly preferring "probable_prime(n) given n = highbit | randrange(1, highbit, 2)" to "probable_prime(n given n = highbit | randrange(1, highbit, 2))"), but it would probably still be simpler overall than attempting to restrict the construct purely to if/elif/while.

Even as a generally available postfix keyword, "given" should still be amenable to the treatment where it could be allowed as a variable name in a non-operator context (since we don't allow two adjacent expressions to imply a function call, it's only prefix keywords that have to be disallowed as names to avoid ambiguity in the parser).

Cheers,

Nick.

--

Nick Coghlan   |   ncog...@gmail.com   |   Brisbane, Australia

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Steven D'Aprano]

On Sun, May 06, 2018 at 02:00:35AM +1000, Nick Coghlan wrote:

> 3. You can stick in an explicit "if True" if you don't need the given
> variable in the filter condition
>
> [(fx**2, fx**3) for x in xs if True given fx = f(x)]

o_O

I'm replying to an email which is a week old. I haven't seen
anyone other than Tim comment on that "if True" boilerplate. Are you
still proposing that? If not, you can ignore the following.


> And then once you've had an entire release where the filter condition was
> mandatory for the comprehension form, allowing the "if True" in "[(fx**2,
> fx**3) for x in xs given fx = f(x)]" to be implicit would be less ambiguous.

I'm sorry, perhaps I'm having another slow day, but I don't see the
ambiguity in the first place.

And I *certainly* do not see how adding in a logically superfluorous
"if True" would make it unambiguous.

For comparison sake, here is the PEP 572 proposal compared to yours:

[((fx := f(x))**2, fx**3) for x in xs]

I'd read that as

for each x in xs, let fx be f of x, return fx squared and fx cube


[(fx**2, fx**3) for x in xs if True given fx = f(x)]

which I'd read as

for each x in xs, if True, return fx squared and fx cube, given fx is f of x

and then wonder why on earth the test is there. Explain to me again why
this boilerplate is necessary, please. Especially since you're already
suggesting that in a future release it could be dropped without changing
the meaning.



[Tim Peters]
> > In
> >
> > if match given match = pattern.search(data):
> >
> > the annoying visual redundancy (& typing) persists.
>
> Right, but that's specific to the case where the desired condition really
> is just "bool(target)".

That really isn't the case.

if result.method() given result = compare(data, arg):

versus the PEP 572 syntax:

if (result := compare(data, arg)).method():

So it certainly isn't just the bool(target) case that is redundant.



--
Steve

[Steven D'Aprano]

On Sat, May 12, 2018 at 08:16:07AM -0700, Neil Girdhar wrote:

> I love given compared with := mainly because
>
> Simpler is better than complex:
> * given breaks a complex statement into two simpler ones,

(Foreshadowing: this argument applies to augmented assignment. See
below.)

I don't see how you justify that statement about "given". I think that
it is "given" which is more complex. Significantly so.

Let's compare the syntax:

target := expr


That is a single, simple expression with a single side-effect: it
assigns the value to <target>. That's it.

Like all expressions, it returns a value, namely the result of "expr".
Like all expressions, you can embed it in other expressions (possibly
wrapping it in parens to avoid precedence issues), or not, as required.

(That surrounding expression can be as simple or complex as you like.)


Now here's Nick's syntax:

target given target = expr

Exactly like the := version above, we can say that this is a single
expression with a single side-effect. Like all expressions, it returns a
value, namely the result of "expr", and like all expressions, you can
embed it in other expressions.

So far the two are precisely the same. There is no difference in the
complexity, because they are exactly the same except for the redundant
and verbose "given" spelling.

But actually, I lied.

Nick's syntax is *much more complicated* than the := syntax. Any
arbitrary expression can appear on the left of "given". It need not
even involve the binding target! So to make a fair comparison, I ought
to compare:

target := expr

which evaluates a single expression, binds it, and returns it, to:

another_expr given target := expr

which evaluates "expr", binds it to "target", evaluates a SECOND
unrelated expression, and returns that.

If you want to argue that this is more useful, then fine, say so. But to
say that it is *simpler* makes no sense to me.

Option 1: evaluate and bind a single expression

Option 2: exactly the same as Option 1, and then evaluate a second
expression

How do you justify that Option 2 "given", which does everything := does
PLUS MORE, is simpler than Option 1?

That's not a rhetorical question.

> which is putting people off in the simple examples shown here (some
> people are annoyed by the extra characters). However, when given is
> used in a list comprehension to prevent having to re-express it as a
> for loop, then two simple statements are easier to understand than one
> complex statement.

I'd like to see an example of one of these list comprehensions that is
simpler written with given. Here's an earlier example, an exponentially
weighted running average:

average = 0
smooth_signal = [(average := (1-decay)*average + decay*x) for x in signal]

assert average == smooth_signal[-1]


I'm not even sure if "given" will support this. Nick is arguing strongly
that bound targets should be local to the comprehension, and so I think
you can't even write this example at all with Nick's scoping rule.

But let's assume that the scoping rule is the same as the above. In that
case, I make it:

average = 0
smooth_signal = [average given average = (1-decay)*average + decay*x) for x in signal]

Is it longer, requiring more typing? Absolutely.

Does it contain a redundantly repetitious duplication of the repeated
target name? Certainly.

But is it simpler? I don't think so.


If you don't like the exponential running average, here's a simple
running total:


total = 0
running_totals = [total := total + x for x in xs]


versus


total = 0
running_totals = [total given total = total + x for x in xs]


If you don't like this example either, please show me an example of an
actual list comp that given makes simpler.

> This is a
> common difference between code written at programming contests versus code
> written by those same software engineers years later at big companies.
> Code that you write for yourself can be compact because you already
> understand it, but code you write professionally is read many many more
> times than it is written. Accessibility is much more important than
> concision.

Ah, nice rhetorical argument: "given" is for professionals, := is a hack
for amateurs and programming contests. Seriously?

Do you use augmented assignment? Your simple versus complex argument for
"given" applies well to augmented assignment.

Augmented assignment combines two conceptual operations:

x = x + 1
- addition (for example)
- assignment

into a single operator:

x += 1


By your argument, augmented assignment is more complex, and we ought to
prefer splitting it into two separate operations x = x + 1 because
that's simpler.

I think I agree that x = x + 1 *is* simpler. We can understand it by
understanding the two parts separately: x+1, followed by assignment.

Whereas += requires us to understand that the syntax not only calls a
dunder method __iadd__ (or __add__ if that doesn't exist), which
potentially can operate in place, but it also does an assignment, all in
one conceptual operation. There's a whole lot of extra complexity there.

That doesn't mean I agree with your conclusion that we ought to prefer
the simpler version, let alone that the complex case (augmented
assignment) is fit only for programming contests and that professionals
ought to choose the simpler one.



--
Steve

[Tim Peters]

Just clarifying a fine point here:

[Steven D'Aprano <st...@pearwood.info>]
> ...

> average = 0
> smooth_signal = [(average := (1-decay)*average + decay*x) for x in signal]
> assert average == smooth_signal[-1]
>
> I'm not even sure if "given" will support this. Nick is arguing strongly
> that bound targets should be local to the comprehension, and so I think
> you can't even write this example at all with Nick's scoping rule.

You can't under Nick's proposal(s), at least not directly (there are
always "tricks"). But it also blows up with UnboundLocalError (for
the "average" in "(1-decay)*average") under the current PEP 572 (the
":=" PEP).

I've proposed to change 572's scoping rules for targets of assignment
expressions appearing in comprehensions so that "it would just work"
instead, but that's getting strong opposition too. My favorite so far
was Nick's (Coghlan's) entertainingly hyperbolic:

"Comprehension scopes are already confusing, so it's OK to dial their
weirdness all the way up to 11" is an *incredibly* strange argument to be
attempting

:-)

The scope issues are logically independent of assignment-expression
spelling, but it's a pretty safe guess Nick is opposed to that example
ever "just working" regardless of spelling, while PEP 572 doesn't
currently support it anyway. Last I heard, Chris (Angelico - the
PEP's author) didn't seem keen on changing it either.

[Carl Smith]

Minor gripe: The ability to articulate Python is not the same as the ability to type Python verbally.

Nobody articulates `def area(width, height): return width * height` as def area, open paren, width, comma, space, height, closed paren...

They would say something like def area as a function of width and height, equal to width by height.

-- Carl Smith

carl....@gmail.com

[Steven D'Aprano]

Part 2.

On Sat, May 12, 2018 at 08:16:07AM -0700, Neil Girdhar wrote:

> I love given compared with := mainly because

[...]

> * Python has a reputation for being working pseudocode, and given reads
> like pseudocode. := needs to be deciphered by comparison especially in the
> complicated cases where multiple := operators are used on one line.

Until you learn and become familiar with a new syntax, there is
generally going to be a period you have to decipher it. I spent a long
time mentally translating list comprehensions into mathematical set
builder notation before it became second nature to me.

Even now, I know people who find decorators and comprehensions
indecipherable. Or at least, so they claim, and they aren't motivated to
bother learning them. Oh well, that's their loss.

Binding-expressions aren't like asynchronous programming, where the
entire coding paradigm is different, and you literally have to think
about your algorithms in another way. Whether you spell the binding
operation

target := expr
expr as target
expr -> target

target given target = expr

let target = expr
: target expr ;

(that last one is stolen from Forth, and should not be taken as a
serious suggestion)

is just a matter of spelling. We'll get used to whatever spelling it is.
Some may be more convenient than others, or more error-prone, or may be
harder to parse, but they're secondary issues. (Important, but still
secondary.) Fundamentally, the operation is the same regardless of the
spelling:

- evaluate an expression
- bind that value to a name
- return the value


(Except of course Nick's "given" suggestion is more complex, since the
returned value is not necessarily the same as the bound value.)

> * there are no difficult mental questions about evaluation order, e.g., in
> a bracketed expression having multiple assignments.

Aren't there just?

x = 1
print( x + x given x = 50 )


Will that print 100, or 51? Brackets ought to make it clearer:

(x + x given x = 50) # this ought to return 100
x + (x given x = 50) # this ought to return 51

but if I leave the brackets out, I have no idea what I'll get.

> Similarly, instead of
> (a.b(a) given a = c.d()) do I write (a.b(a := c.d())) or ((a :=
> c.d()).b(a)) ?

I would expect that your first example is a NameError:

a.b(a := c.d())

since Python evaluates arguments to a method *after* looking up the
method. So that corresponds to:

- look up "a" # NameError, unless you've already got an "a"
- look up "a.b"
- evaluate c.d()
- assign that value to a
- pass that to the a.b method we found earlier


What you probably want is the second version:

(a := c.d()).b(a)

which of course looks like utter crap. It might look better if we use at
least half-way sensible variable names and a more realistic looking
example, instead of obfuscated one-letter names.

(widget := widget_builder.new(*args)).method(widget)

> * it avoids the question of what happens when := is used in a switch: (a
> if (b := c) else d) Sometimes you want the assignment to happen
> unconditionally (a if (b:=c) else d) + b; sometimes you don't. How do you
> force one case or the other?

I think that this argument is really weak. The obvious answer is, if you
don't want the assignment to happen unconditionally, then don't do the
assignment unconditionally. Where you do it depends on when you want the
assignment to take place. There's no mystery here.


# unconditionally pop from a list
(spam if mylist.pop(idx) else eggs) + mylist

# conditionally pop from a list
(mylist.pop(idx) if condition else eggs) + mylist
(spam if condition else mylist.pop(idx)) + mylist


Take your choice of which you want:

(spam if (mylist := expr) else eggs) + mylist
((mylist := expr) if condition else eggs) + mylist
(spam if condition else (mylist := expr)) + mylist

Of course, in the last two cases, you're going to get a NameError when
you try to add mylist if the ternary operator took the wrong branch.
Unless you already defined mylist earlier.

If you want something else, you have to explain what you want.

> given makes it obvious by separating
> assignment from the usage of its assignment target.

This is just a purely mechanical source transformation from one to the
other. Just replace ":" with "mylist given mylist " and you are done.

# unconditional version
(spam if (mylist given mylist = expr) else eggs) + mylist

# conditional versions
((mylist given mylist = expr) if condition else eggs) + mylist
(spam if condition else (mylist given mylist = expr)) + mylist

If you can write the "given" versions, then just do the reverse
transformation and replace the redundant verbosity with a colon.

> Style:
> * it avoids the big style question of when to use and when not to use :=.
> (Even if you ask people not to, people are going to write the
> expression-statement a := b as a synonym for the statement a = b.)

What if they do? Is it really the end of the world if some ex-Pascal
coders or people with an over-developed desire for (foolish) consistency
add a colon to their statement level assignments?

Some people add semi-colons to their statements too. Do we care? No.

But if it aggitates people so much, then I'm perfectly happy with
Guido's suggestion that we simply ban top level binding expressions and
require them to leave the colons out.

> * it looks a lot like the existing Python "for" and "if" clauses, which
> also do in-expression assignments.

"if" clauses do an assignment? Have you borrowed the keys to Guido's
time machine, and are writing from 2025 and Python 4.2? *wink*

I don't think "given" expressions look even remotely similar to either.

for target in iterable:

if condition:

another_expr given target = expr

Aside from "all three use a keyword".


--
Steve

[Stephen J. Turnbull]

David Mertz writes:

> Only the BDFL has a vote with non-zero weight.

"Infinitesimal" != "zero".

Pedantically yours,