[Python-ideas] PEP 671 (late-bound arg defaults), next round of discussion!

[Chris Angelico]

I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
with some additional information about the reference implementation,
and some clarifications elsewhere.

*PEP 671: Syntax for late-bound function argument defaults*

Questions, for you all:

1) If this feature existed in Python 3.11 exactly as described, would
you use it?

2) Independently: Is the syntactic distinction between "=" and "=>" a
cognitive burden?

(It's absolutely valid to say "yes" and "yes", and feel free to say
which of those pulls is the stronger one.)

3) If "yes" to question 1, would you use it for any/all of (a) mutable
defaults, (b) referencing things that might have changed, (c)
referencing other arguments, (d) something else?

4) If "no" to question 1, is there some other spelling or other small
change that WOULD mean you would use it? (Some examples in the PEP.)

5) Do you know how to compile CPython from source, and would you be
willing to try this out? Please? :)

I'd love to hear, also, from anyone's friends/family who know a bit of
Python but haven't been involved in this discussion. If late-bound
defaults "just make sense" to people, that would be highly
informative.

Any and all comments welcomed. I mean, this is python-ideas after
all... bikeshedding is what we do best!

The reference implementation currently has some test failures, which
I'm looking into. I'm probably going to make this my personal default
Python interpreter for a while, to see how things go.

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/UVOQEK7IRFSCBOH734T5GFJOEJXFCR6A/
Code of Conduct: http://python.org/psf/codeofconduct/

[Abdulla Al Kathiri]

> On 1 Dec 2021, at 10:16 AM, Chris Angelico <ros...@gmail.com> wrote:
>
> I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
> with some additional information about the reference implementation,
> and some clarifications elsewhere.
>
> *PEP 671: Syntax for late-bound function argument defaults*
>
> Questions, for you all:
>
> 1) If this feature existed in Python 3.11 exactly as described, would
> you use it?

Yes I will use it.

>
> 2) Independently: Is the syntactic distinction between "=" and "=>" a
> cognitive burden?

No, because it would look like a lambda (if the new lambda syntax were approved), indicating this will be evaluated each time the function is run.

>
> (It's absolutely valid to say "yes" and "yes", and feel free to say
> which of those pulls is the stronger one.)
>
> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> defaults, (b) referencing things that might have changed, (c)
> referencing other arguments, (d) something else?

I will definitely use it for default mutable collections like list, set, dictionary etc. I will also use it to reference things that might have changed. For example, when making callbacks to GUI push buttons, I find myself at the start of the function/callback to be fetching the values from other widgets so we can do something with them. Now those values can be directly passed as late-bound defaults from their respective widgets (e.g., def callback(self, text1 => self.line_edit.text()): …).

>
> 4) If "no" to question 1, is there some other spelling or other small
> change that WOULD mean you would use it? (Some examples in the PEP.)
>

NA

> 5) Do you know how to compile CPython from source, and would you be
> willing to try this out? Please? :)

I haven’t done it from source. I might try to learn how to do it in the next weekend and give it a try.

>
> I'd love to hear, also, from anyone's friends/family who know a bit of
> Python but haven't been involved in this discussion. If late-bound
> defaults "just make sense" to people, that would be highly
> informative.

I will show this to some of my coworkers who are python experts and I will report back.

>
> Any and all comments welcomed. I mean, this is python-ideas after
> all... bikeshedding is what we do best!
>
> The reference implementation currently has some test failures, which
> I'm looking into. I'm probably going to make this my personal default
> Python interpreter for a while, to see how things go.
>
> ChrisA
> _______________________________________________
> Python-ideas mailing list -- python...@python.org
> To unsubscribe send an email to python-id...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at https://mail.python.org/archives/list/python...@python.org/message/UVOQEK7IRFSCBOH734T5GFJOEJXFCR6A/
> Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/EBF27XGKOZPVR2WE46ZF4EWFPY4KU4W2/

[Jeremiah Vivian]

Answering questions:

> 1) If this feature existed in Python 3.11 exactly as described, would
> you use it?

I would definitely use it.

> 2) Independently: Is the syntactic distinction between "=" and "=>" a
> cognitive burden?

No, it isn't much of a cognitive burden.

> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> defaults, (b) referencing things that might have changed, (c)
> referencing other arguments, (d) something else?

Probably all options other than (d).

> 5) Do you know how to compile CPython from source, and would you be
> willing to try this out? Please? :)

Yes I do know how to, and I'm gonna try it out soon enough. I'll reply to this comment once I've done so.

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/62QKLLLO6ISAL3OXASRJEXV73TC23PHZ/

[Steven D'Aprano]

On Wed, Dec 01, 2021 at 07:47:49AM -0000, Jeremiah Vivian wrote:

> > 2) Independently: Is the syntactic distinction between "=" and "=>" a
> > cognitive burden?
> No, it isn't much of a cognitive burden.

You say that now, but if you read function definitions that looked
like this:

def process(func:List->int=>xs=>expression)->int:
...


would you still agree?


--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/ICCQ6VJ7MOXGC7MTMSEISXJVIEXGI2M3/

[Chris Angelico]

On Wed, Dec 1, 2021 at 6:43 PM Abdulla Al Kathiri
<alkathir...@gmail.com> wrote:
> > On 1 Dec 2021, at 10:16 AM, Chris Angelico <ros...@gmail.com> wrote:
> > 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> > defaults, (b) referencing things that might have changed, (c)
> > referencing other arguments, (d) something else?
> I will definitely use it for default mutable collections like list, set, dictionary etc. I will also use it to reference things that might have changed. For example, when making callbacks to GUI push buttons, I find myself at the start of the function/callback to be fetching the values from other widgets so we can do something with them. Now those values can be directly passed as late-bound defaults from their respective widgets (e.g., def callback(self, text1 => self.line_edit.text()): …).
> >

Very interesting. That doesn't normally seem like a function default -
is the callback ever going to be passed two arguments (self and actual
text) such that the default would be ignored?

But, hey, if it makes sense in your code to make it a parameter, sure!

> > 5) Do you know how to compile CPython from source, and would you be
> > willing to try this out? Please? :)
> I haven’t done it from source. I might try to learn how to do it in the next weekend and give it a try.
>
> > I'd love to hear, also, from anyone's friends/family who know a bit of
> > Python but haven't been involved in this discussion. If late-bound
> > defaults "just make sense" to people, that would be highly
> > informative.
> I will show this to some of my coworkers who are python experts and I will report back.

Thank you! All feedback greatly appreciated.

Building CPython from source can be done by following these instructions:

https://devguide.python.org/setup/

Instead of creating your own clone from the pristine master copy,
instead clone my repository at https://github.com/rosuav/cpython and
checkout the pep-671 branch. That'll give you my existing reference
implementation (it's pretty crummy but it mostly works).

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/7VOA7BVTV5UJ24HOPMKO4XK6HW73DH5H/

[Chris Angelico]

On Wed, Dec 1, 2021 at 7:07 PM Chris Angelico <ros...@gmail.com> wrote:

>
> On Wed, Dec 1, 2021 at 6:58 PM Steven D'Aprano <st...@pearwood.info> wrote:
> > You say that now, but if you read function definitions that looked
> > like this:
> >
> > def process(func:List->int=>xs=>expression)->int:
> > ...
> >
> >
> > would you still agree?
> >
>

> I'm not sure what that's supposed to mean.

(To clarify: by "that", I mean "the hypothetical line of code", not
the question you're asking. I'm not sure what the code example is
supposed to mean.)

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/SYGEFUEDPXYCG6C65ASVCCHJDWGPTLBI/

[Chris Angelico]

On Wed, Dec 1, 2021 at 6:58 PM Steven D'Aprano <st...@pearwood.info> wrote:
>

> On Wed, Dec 01, 2021 at 07:47:49AM -0000, Jeremiah Vivian wrote:
>
> > > 2) Independently: Is the syntactic distinction between "=" and "=>" a
> > > cognitive burden?
> > No, it isn't much of a cognitive burden.
>
> You say that now, but if you read function definitions that looked
> like this:
>
> def process(func:List->int=>xs=>expression)->int:
> ...
>
>
> would you still agree?
>

I'm not sure what that's supposed to mean. Is List->int an annotation,
and if so, who's deciding on that syntax? You seem to have used two
different arrows in two different ways:

List->int # presumably an annotation meaning "function that takes
List, returns int"
func:ann=>dflt # late-bound default, completely unnecessary here
xs=>expression # presumably a lambda function
def process(args)->int # return value annotation

Why should List->int and xs=>expression use different arrows? Wouldn't
it be much more reasonable for them to use the same one, whichever
that be? And if that does turn out to be "=>", then yes, I would be
looking at changing PEP 671 to recommend := or =: or something, for
clarity (but still an equals sign with one other symbol next to it).

It's always possible to come up with pathological code. But this is
only really as bad as you describe if it has zero spaces in it.
Otherwise, it's pretty easy to clarify which parts go where:

def process(func: List->int => xs=>expression) -> int:
...

Tada, easy grouping.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/SM2LL7OXRBLRSAAES5NUMMORYOGMOR7U/

[Jeremiah Vivian]

> I'll reply to this comment once I've done so.

I just realized that this message replied on a comment instead of the thread. Anyways, it's a really impressive implementation (hope this statement doesn't spark some sort of debate).

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/F7R5WL7OINCGOC2MQJYYXPDBEWSPDB4V/

[Chris Angelico]

On Wed, Dec 1, 2021 at 7:12 PM Jeremiah Vivian
<nohacking...@gmail.com> wrote:
>
> > I'll reply to this comment once I've done so.
> I just realized that this message replied on a comment instead of the thread. Anyways, it's a really impressive implementation (hope this statement doesn't spark some sort of debate).
>

I'm.... going to take that at face value and interpret it as a
compliment. Thank you. :)

If you intended that to be taken differently, then yeah, you're
probably not wrong there too - it is, in a sense, quite... impressive.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/AHX6OIPPL355GGCBQDSVJM6G7KEVVDUR/

[Abdulla Al Kathiri]

> On 1 Dec 2021, at 12:01 PM, Chris Angelico <ros...@gmail.com> wrote:
>
> On Wed, Dec 1, 2021 at 6:43 PM Abdulla Al Kathiri
> <alkathir...@gmail.com> wrote:
>>> On 1 Dec 2021, at 10:16 AM, Chris Angelico <ros...@gmail.com> wrote:
>>> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
>>> defaults, (b) referencing things that might have changed, (c)
>>> referencing other arguments, (d) something else?
>> I will definitely use it for default mutable collections like list, set, dictionary etc. I will also use it to reference things that might have changed. For example, when making callbacks to GUI push buttons, I find myself at the start of the function/callback to be fetching the values from other widgets so we can do something with them. Now those values can be directly passed as late-bound defaults from their respective widgets (e.g., def callback(self, text1 => self.line_edit.text()): …).
>>>
>
> Very interesting. That doesn't normally seem like a function default -
> is the callback ever going to be passed two arguments (self and actual
> text) such that the default would be ignored?

Yeah. Let’s say the callback prints the text on the main window console. I could use the same function to print something on the console not related to the default widget changing text. Maybe another push button that prints literal “WOW”. If I made the second argument a default widget (def callback(self, line_edit=self.line_edit):...) and then call line_edit.text() inside the function, I would be able to pass any other widget that has the method text() but I wouldn’t be able to pass a literal text.

Message archived at https://mail.python.org/archives/list/python...@python.org/message/HE3ST52STGZQ3OJRVEQ2DGEPCM3RQHZF/

[Chris Angelico]

On Wed, Dec 1, 2021 at 8:18 PM Abdulla Al Kathiri

<alkathir...@gmail.com> wrote:
>
>
>
> > On 1 Dec 2021, at 12:01 PM, Chris Angelico <ros...@gmail.com> wrote:
> >
> > On Wed, Dec 1, 2021 at 6:43 PM Abdulla Al Kathiri
> > <alkathir...@gmail.com> wrote:
> >>> On 1 Dec 2021, at 10:16 AM, Chris Angelico <ros...@gmail.com> wrote:
> >>> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> >>> defaults, (b) referencing things that might have changed, (c)
> >>> referencing other arguments, (d) something else?
> >> I will definitely use it for default mutable collections like list, set, dictionary etc. I will also use it to reference things that might have changed. For example, when making callbacks to GUI push buttons, I find myself at the start of the function/callback to be fetching the values from other widgets so we can do something with them. Now those values can be directly passed as late-bound defaults from their respective widgets (e.g., def callback(self, text1 => self.line_edit.text()): …).
> >>>
> >
> > Very interesting. That doesn't normally seem like a function default -
> > is the callback ever going to be passed two arguments (self and actual
> > text) such that the default would be ignored?
> Yeah. Let’s say the callback prints the text on the main window console. I could use the same function to print something on the console not related to the default widget changing text. Maybe another push button that prints literal “WOW”. If I made the second argument a default widget (def callback(self, line_edit=self.line_edit):...) and then call line_edit.text() inside the function, I would be able to pass any other widget that has the method text() but I wouldn’t be able to pass a literal text.
>

Ahh gotcha. Then yes, you have a function that takes up to two
parameters (or one if you don't count self), and yes, omitting the
second argument will have the same effect as fetching the text from
that control. So, sounds like a great use for it!

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/7GUNW4R3TSJC6UXQ5D4P7QQ3XBF75NK2/

[Neil Girdhar]

On Wednesday, December 1, 2021 at 1:18:33 AM UTC-5 Chris Angelico wrote:

I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
with some additional information about the reference implementation,
and some clarifications elsewhere.

*PEP 671: Syntax for late-bound function argument defaults*

Questions, for you all:

1) If this feature existed in Python 3.11 exactly as described, would
you use it?

No, I would avoid for the sake of readers.  Anyway, it can't be used in any public projects until thy drop support for 3.10, which is many years off.

Also, I think this question is quite the biased sample on python-ideas. Please consider asking this to less advanced python users, e.g., reddit.com/r/python or learnpython. 

2) Independently: Is the syntactic distinction between "=" and "=>" a
cognitive burden?

(It's absolutely valid to say "yes" and "yes", and feel free to say
which of those pulls is the stronger one.)

 Yes.

3) If "yes" to question 1, would you use it for any/all of (a) mutable
defaults, (b) referencing things that might have changed, (c)
referencing other arguments, (d) something else?

4) If "no" to question 1, is there some other spelling or other small
change that WOULD mean you would use it? (Some examples in the PEP.)

No. 

 

5) Do you know how to compile CPython from source, and would you be
willing to try this out? Please? :)

I'd love to hear, also, from anyone's friends/family who know a bit of
Python but haven't been involved in this discussion. If late-bound
defaults "just make sense" to people, that would be highly
informative.

Any and all comments welcomed. I mean, this is python-ideas after
all... bikeshedding is what we do best! 

 

This PEP has a lot of interesting ideas.  I still think that none-aware operators (PEP 505) are an easier, more readable general solution to binding calculated default values to arguments succinctly.  I think the problems with this idea include:

* The caller cannot explicitly ask for default behaviour except by omitting the parameter.  This can be very annoying to set up when the parameter values are provided from other places, e.g.,

if need_x:

  # do lots of stuff

  x = whatever

else:

  # do more

  x = None

f(x=x, ...)  # easy, but with this PEP, you would have to find a way to remove x from the parameter list.  Typically, removing a parameter from a dynamically-created parameter list is hard.

* The function code becomes unreadable if the parameter-setting code is long.  Having a long piece of code inside the function after "if parameter is None" is just fine.  Having none-aware operators would make such code more succinct.

* People nearly always avoid writing code in the parameter defaults themselves, and this new practice adds a lot of cognitive load.  E.g., people rarely write:

def f(x: int = 1+g()) -> None: ...

Parameter lists are already busy enough with parameter names, annotations, and defaults.  We don't need to encourage this practice.

In short, I think this is a creative idea, a great exploration.  While optional parameters are common, and some of them have defaults that are calculated inside the function, my feeling is that people will continue to set their values inside the function.

Best,

Neil

[Steven D'Aprano]

On Wed, Dec 01, 2021 at 07:07:20PM +1100, Chris Angelico wrote:

> > def process(func:List->int=>xs=>expression)->int:
> > ...

> I'm not sure what that's supposed to mean.

You did a pretty good job of working it out :-)

> Is List->int an annotation,
> and if so, who's deciding on that syntax? You seem to have used two
> different arrows in two different ways:

Exactly my point. Beyond your arrow syntax for default parameters, and
the existing return annotation use, there are two other hypothetical
proposals for arrow syntax on the table:

- using `->` as an alias for typing.Callable;

- using `=>` as a more compact lambda;

See here:

https://lwn.net/Articles/847960/

I rate the first one (the typing.Callable alias) as extremely likely. It
is hard to annotate parameters which are functions, and the typing
community is actively working on making annotations easier. So my gut
feeling is that it is only a matter of time before we have annotations
like `func:List->int`. And probably closer to 3.11 than 3.20 :-)

The second, the compact lambda, I rate as only moderately likely. Guido
seems to like it, but whether somebody writes a PEP and the Steering
Council accepts it is uncertain. But it is something that does keep
coming up, and I think we should consider it as a possibility.

Even if `=>` doesn't get used for compact lambda, it is an obvious
syntax to use for *something* and so we should consider how late-binding
will look, not just with existing syntax, but also with *likely* future
syntax.

(When you build a road, you don't plan for just the population you have
now, but also the population you are likely to have in the future.)

I don't expect anyone to take into account *arbitrary* and unpredictable
future syntax, but it is wise to take into account future syntax that
we're already planning or that seems likely.

> List->int # presumably an annotation meaning "function that takes
> List, returns int"

Correct.

> func:ann=>dflt # late-bound default, completely unnecessary here

Come on Chris, how can you say that it is "completely unnecessary"?
Unless that's an admission that late-bound defaults are all
unnecessary... *wink*

There is a difference between these two:

def func(arg=lambda a: expression):
...

and this:

def func(arg=None):
if arg is None:
arg = lambda a: expression
...

therefore there will be a difference between:

def func(arg=lambda a: expression):
def func(arg=>lambda a: expression):

If nothing else, in the first case (early binding) you get the same
function object every time. In the second, you get a freshly made
function object each time. Since function objects are mutable (they have
a writable `__dict__` that's a visible difference even if the bodies are
identical. And they may not be.

But even if it is unnecessary, it will still be permitted, just as we
will be able to write:

# Now this *actually is* totally unnecessary use of late-binding
def func(arg=>None):

> xs=>expression # presumably a lambda function
> def process(args)->int # return value annotation
>
> Why should List->int and xs=>expression use different arrows?

Because they do different things. To avoid confusion.

Chris, you wrote the PEP for the walrus operator. Why should the
assignment operator use a different symbol from the assignment
statement? Same reason that typing.Callable and lambda will likely use
different arrows.

Anyway, if you disagree, take it up with Guido, it was his suggestion to
use different arrows :-P

> Wouldn't
> it be much more reasonable for them to use the same one, whichever
> that be? And if that does turn out to be "=>", then yes, I would be
> looking at changing PEP 671 to recommend := or =: or something, for
> clarity (but still an equals sign with one other symbol next to it).

Oh great, now you're going to conflict with walrus...

def process(obj:Union[T:=something, List[T]]:=func(x:=expression)+x)->T:

We ought to at least try to avoid clear and obvious conflicts between
new and existing syntax.

Using `:=` is even worse than `=>`, and `=:` is just *begging* to
confuse newcomers "why do we have THREE different assignment symbols?"

> It's always possible to come up with pathological code. But this is
> only really as bad as you describe if it has zero spaces in it.
> Otherwise, it's pretty easy to clarify which parts go where:
>
> def process(func: List->int => xs=>expression) -> int:
> ...

"My linter complains about spaces around operators! Take them out!"

Or maybe we should put more in? Spaces are optional.

def process(func : List -> int => xs => expression) -> int:
...

I'm not saying that an experienced Pythonista who is a careful reader
can't work out what the meaning is. It's not *ambiguous* syntax to a
careful reader. But it's *confusing* syntax to somebody who may not be
as careful and experienced as you, or is coding late at night, or in a
hurry, or coding while tired and emotional or distracted.

We should prefer to avoid confusable syntax when we can. The interpreter
can always disambiguate assignment as an expression from assignment as a
statement, but we still chose to make it easy on the human reader by
using distinct symbols.

def process(arg>=expression)

would be totally unambiguous to the intepreter, but I trust you would
reject that because it reads like "greater than" to the human reader.


--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/LMTIT4G2ASTKHZYE7HBUAYBXE7ASEOBO/

[Chris Angelico]

On Wed, Dec 1, 2021 at 8:57 PM Neil Girdhar <miste...@gmail.com> wrote:
>
>
>
> On Wednesday, December 1, 2021 at 1:18:33 AM UTC-5 Chris Angelico wrote:
>>
>> I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
>> with some additional information about the reference implementation,
>> and some clarifications elsewhere.
>>
>> *PEP 671: Syntax for late-bound function argument defaults*
>>
>> Questions, for you all:
>>
>> 1) If this feature existed in Python 3.11 exactly as described, would
>> you use it?
>
> No, I would avoid for the sake of readers. Anyway, it can't be used in any public projects until thy drop support for 3.10, which is many years off.
>
> Also, I think this question is quite the biased sample on python-ideas. Please consider asking this to less advanced python users, e.g., reddit.com/r/python or learnpython.

I know it is :) I never expected to get an unbiased sample (from
anywhere, least of all here).

> This PEP has a lot of interesting ideas. I still think that none-aware operators (PEP 505) are an easier, more readable general solution to binding calculated default values to arguments succinctly. I think the problems with this idea include:
> * The caller cannot explicitly ask for default behaviour except by omitting the parameter. This can be very annoying to set up when the parameter values are provided from other places, e.g.,
> if need_x:
> # do lots of stuff
> x = whatever
> else:
> # do more
> x = None
> f(x=x, ...) # easy, but with this PEP, you would have to find a way to remove x from the parameter list. Typically, removing a parameter from a dynamically-created parameter list is hard.
>

You can always do that with *args or **kwargs if that's what you need,
and that actually DOES represent the concept of "don't pass this
argument". Passing None is a hack that doesn't actually mean "don't
pass this argument", and if you're depending on that behaviour, it's
been locked in as the function's API, which causes problems if None
ever becomes a meaningful non-default value.

None-aware operators are solving a different problem. They allow you
to accept None, but then simplify the replacement of it with some
other value. This proposal allows you to leave None out of the
equation altogether. And it only works if None isn't a valid value.

> * The function code becomes unreadable if the parameter-setting code is long. Having a long piece of code inside the function after "if parameter is None" is just fine. Having none-aware operators would make such code more succinct.
>

Again, that only works if None is not a valid parameter value, and PEP
505 doesn't generalize to other sentinels at all.

If the code is too long, don't put it into the parameter default. But
"if x is None: x = []" can easily be replaced with "x=>[]" and the
parameters will actually become shorter.

ANY feature can be used badly. You can daisy-chain everything into a
gigantic expression with horrendous abuses of lambda functions, but
that doesn't mean you should. It also doesn't mean that lambda
functions are bad :)

> * People nearly always avoid writing code in the parameter defaults themselves, and this new practice adds a lot of cognitive load. E.g., people rarely write:
> def f(x: int = 1+g()) -> None: ...
> Parameter lists are already busy enough with parameter names, annotations, and defaults. We don't need to encourage this practice.
>

They don't become any more noisy by being able to use defaults that
aren't constant. The main goal is to put the function header in the
function header, and the function body in the function body, instead
of having the function body doing part of the work of the header :)

> In short, I think this is a creative idea, a great exploration. While optional parameters are common, and some of them have defaults that are calculated inside the function, my feeling is that people will continue to set their values inside the function.
>

In some cases, they certainly will. And that's not a problem. But for
the cases where it's better to put it in the header, it's better to
have that option than to restrict it for an arbitrary technical
limitation.

Some languages don't have ANY function defaults. They simply have
"optional arguments", where an omitted optional argument will always
and only have a specific null value. Python allows you to show what
the default REALLY is, and that's an improvement. I want to be able to
show what the default is, even if that is defined in a non-constant
way.

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/23E62MLZH3ZMRYWTYPXDKG2LWT6X3G2V/

[Chris Angelico]

On Wed, Dec 1, 2021 at 9:09 PM Steven D'Aprano <st...@pearwood.info> wrote:
>
> On Wed, Dec 01, 2021 at 07:07:20PM +1100, Chris Angelico wrote:
>
> > > def process(func:List->int=>xs=>expression)->int:
> > > ...
>
> > I'm not sure what that's supposed to mean.
>
> You did a pretty good job of working it out :-)

Okay, cool, thanks.

> Exactly my point. Beyond your arrow syntax for default parameters, and
> the existing return annotation use, there are two other hypothetical
> proposals for arrow syntax on the table:
>
> - using `->` as an alias for typing.Callable;
>
> - using `=>` as a more compact lambda;

That confuses me. Why would they use different arrows?

If we have a compact lambda syntax, why can't actual functions be used
as annotations to represent functions? I believe there's plans to have
[str] mean List[str], which makes a lot of sense. Why not have "lambda
str: int" or "str=>int" be an annotation too?

But okay. Supposing that annotations have to use one arrow and lambda
functions use another, then yes, this is a good reason to use "=:" for
late-bound defaults. It's not that big a change.

> > func:ann=>dflt # late-bound default, completely unnecessary here
>
> Come on Chris, how can you say that it is "completely unnecessary"?
> Unless that's an admission that late-bound defaults are all
> unnecessary... *wink*
>
> There is a difference between these two:
>
> def func(arg=lambda a: expression):
> ...
>
> and this:
>
> def func(arg=None):
> if arg is None:
> arg = lambda a: expression
> ...
>
> therefore there will be a difference between:
>
> def func(arg=lambda a: expression):
> def func(arg=>lambda a: expression):
>
> If nothing else, in the first case (early binding) you get the same
> function object every time. In the second, you get a freshly made
> function object each time. Since function objects are mutable (they have
> a writable `__dict__` that's a visible difference even if the bodies are
> identical. And they may not be.

Yyyyyes. Okay. There is a technical way in which you might want this
alternate behaviour. Is that REALLY something you're planning on doing
- having a function which takes another function as an argument, and
which behaves differently based on whether it's given the same
function every time or multiple different functions with the same
behaviour?

> But even if it is unnecessary, it will still be permitted, just as we
> will be able to write:
>
> # Now this *actually is* totally unnecessary use of late-binding
> def func(arg=>None):

Yes. That one is absolutely unnecessary, since there is no way you'll
ever get back a different result. Except that you could then mutate
func's dunders and change the behaviour. So in a purely technical
sense, nothing can be called "unnecessary".

Now, in real terms: late-binding a lambda function with no default
arguments seems like a pretty pointless thing to do. I stand by my
original statement :) And I also stand by my original statement that
proper use of the space bar can not only improve readability, it can
also help you find a date with space girls and space guys... or maybe
that last part only works on Mars.

> > xs=>expression # presumably a lambda function
> > def process(args)->int # return value annotation
> >
> > Why should List->int and xs=>expression use different arrows?
>
> Because they do different things. To avoid confusion.
>
> Chris, you wrote the PEP for the walrus operator. Why should the
> assignment operator use a different symbol from the assignment
> statement? Same reason that typing.Callable and lambda will likely use
> different arrows.
>
> Anyway, if you disagree, take it up with Guido, it was his suggestion to
> use different arrows :-P

Fair enough, but typing also started out with List[str] and is now
looking at using list[str] and, I think, [str], because it's more
consistent :)

> > Wouldn't
> > it be much more reasonable for them to use the same one, whichever
> > that be? And if that does turn out to be "=>", then yes, I would be
> > looking at changing PEP 671 to recommend := or =: or something, for
> > clarity (but still an equals sign with one other symbol next to it).
>
> Oh great, now you're going to conflict with walrus...
>
> def process(obj:Union[T:=something, List[T]]:=func(x:=expression)+x)->T:
>
> We ought to at least try to avoid clear and obvious conflicts between
> new and existing syntax.
>
> Using `:=` is even worse than `=>`, and `=:` is just *begging* to
> confuse newcomers "why do we have THREE different assignment symbols?"

If you look at every way the equals sign is used in Python grammar,
there are a lot of subtle differences. The difference between "f(x=1)"
and "x=1" doesn't bother people, and those are exactly the same
symbol. There are only so many symbols that we can type on everyone's
keyboards.

But you're right, and that's why I would very much like to reuse an
existing symbol rather than create new ones.

> > It's always possible to come up with pathological code. But this is
> > only really as bad as you describe if it has zero spaces in it.
> > Otherwise, it's pretty easy to clarify which parts go where:
> >
> > def process(func: List->int => xs=>expression) -> int:
> > ...
>
> "My linter complains about spaces around operators! Take them out!"

Yes, take your linter out the back and execute it :)

> Or maybe we should put more in? Spaces are optional.
>
> def process(func : List -> int => xs => expression) -> int:
> ...
>
> I'm not saying that an experienced Pythonista who is a careful reader
> can't work out what the meaning is. It's not *ambiguous* syntax to a
> careful reader. But it's *confusing* syntax to somebody who may not be
> as careful and experienced as you, or is coding late at night, or in a
> hurry, or coding while tired and emotional or distracted.
>
> We should prefer to avoid confusable syntax when we can. The interpreter
> can always disambiguate assignment as an expression from assignment as a
> statement, but we still chose to make it easy on the human reader by
> using distinct symbols.
>
> def process(arg>=expression)
>
> would be totally unambiguous to the intepreter, but I trust you would
> reject that because it reads like "greater than" to the human reader.
>

Actually that, to me, is no different from the other options - it's
still an arrow (but now more like a funnel than an arrow, which is a
bit weird). It's no more a greater-than than "=>" is.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/WFOFQW2XSVBE43LV6CR6RIFXSQ2ABRFQ/

[Neil Girdhar]

  Of course you're right: passing None is a hack.  I would happily have supported this proposal to eliminate it if you had made this argument 15 years ago :)  But it's now idiomatic Python that pervades practically every Python interface.  A possibly None value is literally annotated as typing.Optional.  The semantics are baked into the language.

 

and if you're depending on that behaviour, it's
been locked in as the function's API, which causes problems if None
ever becomes a meaningful non-default value.

True.

None-aware operators are solving a different problem. They allow you
to accept None, but then simplify the replacement of it with some
other value. This proposal allows you to leave None out of the
equation altogether. And it only works if None isn't a valid value.

Exactly.

 

> * The function code becomes unreadable if the parameter-setting code is long.  Having a long piece of code inside the function after "if parameter is None" is just fine.  Having none-aware operators would make such code more succinct.
>

Again, that only works if None is not a valid parameter value, and PEP
505 doesn't generalize to other sentinels at all.

If the code is too long, don't put it into the parameter default. But
"if x is None: x = []" can easily be replaced with "x=>[]" and the
parameters will actually become shorter.

Right, so my point here is that this proposal has limited scope.  What I was trying to get at was that this proposal doesn't completely supplant the use of None as a sentinel since some default-setting code blocks should still be written using None:

* long code blocks,

* code blocks that need access to other (possibly optional) parameters, and 

* code blocks that need access to some other quantities that are calculated within the function.

If this proposal would supplant all use of None for optional parameters, then I would be much more excited about it.  I could buy into the dream of a future Python with no "typing.Optional" parameters.  But that's not what this proposal sells.  It's more modest.  It sells simplicity in a few special cases at the cost of complexity.  That's not necessarily bad, but my feeling is that the complexity isn't worth it.

ANY feature can be used badly. You can daisy-chain everything into a
gigantic expression with horrendous abuses of lambda functions, but
that doesn't mean you should. It also doesn't mean that lambda
functions are bad :)

 

You're right :). I was just pointing out limited scope.

> * People nearly always avoid writing code in the parameter defaults themselves, and this new practice adds a lot of cognitive load.  E.g., people rarely write:
> def f(x: int = 1+g()) -> None: ...
> Parameter lists are already busy enough with parameter names, annotations, and defaults.  We don't need to encourage this practice.
>

They don't become any more noisy by being able to use defaults that
aren't constant. The main goal is to put the function header in the
function header, and the function body in the function body, instead
of having the function body doing part of the work of the header :)

I understand your point, but I think this is begging the question.  Of course you think that "the function body is doing the work of the header".  My intuition is that any code that is run after the function is called should appear inside the function body.  But that's because you've internalized the consequences of this PEP and I haven't.  From my point of view, it's one more thing I need to check for when a function isn't doing what I expect.

 

> In short, I think this is a creative idea, a great exploration.  While optional parameters are common, and some of them have defaults that are calculated inside the function, my feeling is that people will continue to set their values inside the function.
>

In some cases, they certainly will. And that's not a problem. But for
the cases where it's better to put it in the header, it's better to
have that option than to restrict it for an arbitrary technical
limitation.

Some languages don't have ANY function defaults. They simply have
"optional arguments", where an omitted optional argument will always
and only have a specific null value. Python allows you to show what
the default REALLY is, and that's an improvement. I want to be able to
show what the default is, even if that is defined in a non-constant
way.

Fair points :)

 

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/23E62MLZH3ZMRYWTYPXDKG2LWT6X3G2V/
Code of Conduct: http://python.org/psf/codeofconduct/

--

---
You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-ideas/cz3I79-jyp0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to python-ideas...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python-ideas/CAPTjJmpcUxSOCGfxyDFPU8G-NXHq_59dXAA_t2-YeFbtw7dxyw%40mail.gmail.com.

[André Roberge]

On Wed, Dec 1, 2021 at 2:17 AM Chris Angelico <ros...@gmail.com> wrote:

I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
with some additional information about the reference implementation,
and some clarifications elsewhere.

*PEP 671: Syntax for late-bound function argument defaults*

Questions, for you all:

1) If this feature existed in Python 3.11 exactly as described, would
you use it?

Currently, I'm thinking "absolutely not". 

However, I thought the same about the walrus operator and I now miss not being able to use it in a program that includes support for Python 3.6 and where I have literally dozens of places where I would use it if I could.

 

2) Independently: Is the syntactic distinction between "=" and "=>" a
cognitive burden?

Yes.

I really think that using a keyword like defer, or from_calling_scope ;-), would significantly reduce the cognitive burden.

4) If "no" to question 1, is there some other spelling or other small
change that WOULD mean you would use it? (Some examples in the PEP.)

*Perhaps* if a keyword would be used instead of symbols, I might reconsider.

I find the emphasis of trying to cram too much information in single lines of code to be really a burden. Many years ago, I argued very unsuccessfully for using a 'where:' code block for annotations.  (To this day, I still believe it would make the code much more readable, at the cost of a slight duplication.)  Using what is at first glance a cryptic operator like => for late binding is not helping readability, especially when type annotations are thrown in the mix.

Aside: at the same time, I can see how using => instead of lambda as a potential win in readability, including for beginners.

5) Do you know how to compile CPython from source, and would you be
willing to try this out? Please? :)

Sorry, I'm not interested enough at this point but, given the amount of work you put into this, I decided that the least I could do is provide feedback rather than be a passive reader.

André Roberge

[Chris Angelico]

On Wed, Dec 1, 2021 at 10:30 PM André Roberge <andre....@gmail.com> wrote:
> On Wed, Dec 1, 2021 at 2:17 AM Chris Angelico <ros...@gmail.com> wrote:
>>
>> I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
>> with some additional information about the reference implementation,
>> and some clarifications elsewhere.
>>
>> *PEP 671: Syntax for late-bound function argument defaults*
>>
>> Questions, for you all:
>>
>> 1) If this feature existed in Python 3.11 exactly as described, would
>> you use it?
>>
> Currently, I'm thinking "absolutely not".
>
> However, I thought the same about the walrus operator and I now miss not being able to use it in a program that includes support for Python 3.6 and where I have literally dozens of places where I would use it if I could.

Very fair :)

>> 2) Independently: Is the syntactic distinction between "=" and "=>" a
>> cognitive burden?
>
> Yes.
> I really think that using a keyword like defer, or from_calling_scope ;-), would significantly reduce the cognitive burden.

Also fair. I'm not a fan of keywords for this sort of thing, since it
implies that you could do this:

def f(x=defer []): ...

dflt = defer []
def f(x=dflt): ...

which is a completely different proposal (eg it would be evaluated
only when you "touch" that, rather than being guaranteed to be
evaluated before the first line of the function body). That's why I
want to adorn the equals sign and nothing else.

>> 4) If "no" to question 1, is there some other spelling or other small
>> change that WOULD mean you would use it? (Some examples in the PEP.)
>
>
> *Perhaps* if a keyword would be used instead of symbols, I might reconsider.
>
> I find the emphasis of trying to cram too much information in single lines of code to be really a burden. Many years ago, I argued very unsuccessfully for using a 'where:' code block for annotations. (To this day, I still believe it would make the code much more readable, at the cost of a slight duplication.) Using what is at first glance a cryptic operator like => for late binding is not helping readability, especially when type annotations are thrown in the mix.
>
> Aside: at the same time, I can see how using => instead of lambda as a potential win in readability, including for beginners.

It's interesting how different people's views go on that sort of
thing. It depends a lot on how much people expect to use something.
Features you use a lot want to have short notations, features you
seldom use are allowed to have longer words.

>> 5) Do you know how to compile CPython from source, and would you be
>> willing to try this out? Please? :)
>
>
> Sorry, I'm not interested enough at this point but, given the amount of work you put into this, I decided that the least I could do is provide feedback rather than be a passive reader.

That's absolutely fine. I don't by any means expect everyone to be
able or willing to compile CPython. Feedback is absolutely
appreciated, and I asked the question with full expectation of getting
a few "No" responses :)

Thank you for taking the time to respond. Thoughtful feedback is
incredibly helpful.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/WTU355AA46KWI6QJNMC3HR52SEEXQS65/

[Matt del Valle]

1) If this feature existed in Python 3.11 exactly as described, would
you use it?

Absolutely!

 

2) Independently: Is the syntactic distinction between "=" and "=>" a
cognitive burden?

Nope, I don't think this is an undue cognitive burden. If anything I think the symmetry between the proposed '=>' syntax and the arrow syntax for lambdas in other languages (potentially even in python in the future) reduces the cognitive burden significantly, given the there is an equivalent symmetry with their semantics (in both cases the code is being evaluated later when something is called).

Steven gave the following example of a function signature that would be difficult to visually parse if this proposal and arrow lambdas were accepted:

def process(func:List->int=>xs=>expression)->int:

And while I agree that it does sort of stop you in your tracks when you see this, I think there are a couple of reasons why this is not as big of a problem as it appears.

Firstly, I think if you're writing this sort of code, you can improve the legibility a lot by using appropriate spacing and adding parentheses around the lambda arguments (even if they're unnecessary for a lambda with only a single argument, like in Steven's example):

def process(func: (List) -> int => (xs) => expression) -> int:

Personally I find this much easier to visually parse since I'm used to seeing lambdas in the form:

(*args) => expr

So my brain just kind of groups `(List) -> int` and `(xs) => expression` into atomic chunks fairly effortlessly. And this is before even mentioning the *massive* cognitive assistance provided by syntax highlighting :)

But at this point I would also like to point out that code like this is probably going to be vanishingly rare in the wild. The majority of use-cases for late-bound defaults as far as I can tell are going to be for mutable defaults (and maybe some references to early-bound arguments). I don't like the idea of compromising on what is a very clean and intuitive syntax in 99% of cases to cater to the hypothetical 1% of cases where it admittedly takes a bit more cognitive effort to parse.

I agree with the statement Chris made above where he mentioned that the same symbols sometimes have different meanings based on context, and that's okay. For example:

def some_func(some_arg: int) -> int: ...

def another_func(another_arg: int = some_func(some_arg=3)) -> int: ...

In the second function definition above you've got the '=' symbol pulling double-duty both as declaring an argument for a function def and as a keyword arg for a function call in the same statement. I think this is perfectly fine, and I think the same thing is true of the => symbol being used for both lambdas and late-bound arguments, especially given they are semantically related.

3) If "yes" to question 1, would you use it for any/all of (a) mutable
defaults, (b) referencing things that might have changed, (c)
referencing other arguments, (d) something else?

(a) is the primary use-case I can see myself using this feature for. I'm not sure what you mean by (b). I can also definitely see some situations where (c) would be dead useful, though I don't think this would come up in the sort of code I write as much as (a). Still, when the use-case for (c) did present itself, I would be *extremely* grateful to have late-bound defaults to reach for.

4) If "no" to question 1, is there some other spelling or other small
change that WOULD mean you would use it? (Some examples in the PEP.)

Not relevant, since my strong preference for late-bound default syntax is the '=>' symbol.

[David Mertz, Ph.D.]

On Wed, Dec 1, 2021 at 1:18 AM Chris Angelico <ros...@gmail.com> wrote:

1) If this feature existed in Python 3.11 exactly as described, would
you use it?

No.

... except in the sense that as I trainer I have to teach the warts in Python, and would need to warn students they might see that.

 

2) Independently: Is the syntactic distinction between "=" and "=>" a
cognitive burden?

YES! 

A few weeks later than the prior long discussion that I read in full, it took a triple take not to read it as >= (which would mean something syntactical in many cases, just not what is intended).

 

3) If "yes" to question 1, would you use it for any/all of (a) mutable
defaults, (b) referencing things that might have changed, (c)
referencing other arguments, (d) something else?

I would always recommend against its use if I had any influence on code review.

 

4) If "no" to question 1, is there some other spelling or other small
change that WOULD mean you would use it? (Some examples in the PEP.)

Yes, the delay/later/defer keyword approach is not confusing, and does not preempt a later feature that would actually be worth having.

5) Do you know how to compile CPython from source, and would you be
willing to try this out? Please? :)

I do know how, but it is unlikely I'll have time.

--

Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.

[Paul Moore]

On Wed, 1 Dec 2021 at 06:19, Chris Angelico <ros...@gmail.com> wrote:
>
> I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
> with some additional information about the reference implementation,
> and some clarifications elsewhere.
>
> *PEP 671: Syntax for late-bound function argument defaults*
>
> Questions, for you all:
>
> 1) If this feature existed in Python 3.11 exactly as described, would
> you use it?

Probably not. Mainly because I don't have any real use for it rather
than because I have any inherent problem with it (but see below, the
more I thought about it the more uncomfortable with it I became).

> 2) Independently: Is the syntactic distinction between "=" and "=>" a
> cognitive burden?
>
> (It's absolutely valid to say "yes" and "yes", and feel free to say
> which of those pulls is the stronger one.)

Not especially. The idea of having both late-bound and early-bound
parameters is probably more of a cognitive burden than the syntax.

> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> defaults, (b) referencing things that might have changed, (c)
> referencing other arguments, (d) something else?

N/A, except to say that when you enumerate the use cases like this,
none of them even tempt me to use this feature. I think that the only
thing I might use it for is to make it easier to annotate defaults (as
f(a: list[int] => []) rather than as f(a: list[int] | None = None).

So I'll revise my answer to (1) and say that I *might* use this, but
only in a way it wasn't intended to be used in, and mostly because I
hate how verbose it is to express optional arguments in type
annotations. (And the fact that the annotation exposes the sentinel
value, even when you want it to be opaque). I hope I don't succumb and
do that, though ;-)

> 4) If "no" to question 1, is there some other spelling or other small
> change that WOULD mean you would use it? (Some examples in the PEP.)

Not really. It addresses a wart in the language, but on consideration,
it feels like the cure is no better than the disease.

> 5) Do you know how to compile CPython from source, and would you be
> willing to try this out? Please? :)

Sorry, I really don't have time to, in the foreseeable future. If I
did have time, one thing I would experiment with is how this interacts
with typing and tools like pyright and mypy (yes, I know type checkers
would need updating for the new syntax, so that would mostly be a
thought experiment) - as I say, I'd expect to annotate a function with
an optional list argument defaulting to an empty list as f(a:
list[int] => []), which means that __annotations__ needs to
distinguish between this case and f(a: list[int]) with no default.

>>> def f(a: list[int]): pass
...
>>> f.__annotations__
{'a': list[int]}

>>> def f(a: list[int] => []): pass
...
>>> f.__annotations__
???

> I'd love to hear, also, from anyone's friends/family who know a bit of
> Python but haven't been involved in this discussion. If late-bound
> defaults "just make sense" to people, that would be highly
> informative.

Sorry, I don't have any feedback like that. What I can say, though, is
I'd find it quite hard to express the question, in the sense that I'd
struggle to explain the difference between early and late bound
parameters to a non-expert, much less explain why we need both. I'd
probably just say "it's short for a default of None and a check" which
doesn't really capture the point...

> Any and all comments welcomed. I mean, this is python-ideas after
> all... bikeshedding is what we do best!

I hope this was useful feedback.
Paul

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/BMJLEOHEA7MYC5IPYUSXELPRWGW4VG6M/

[André Roberge]

On Wed, Dec 1, 2021 at 7:51 AM Chris Angelico <ros...@gmail.com> wrote:

On Wed, Dec 1, 2021 at 10:30 PM André Roberge <andre....@gmail.com> wrote:

>> 2) Independently: Is the syntactic distinction between "=" and "=>" a
>> cognitive burden?
>
> Yes.
> I really think that using a keyword like defer, or from_calling_scope ;-), would significantly reduce the cognitive burden.

Also fair. I'm not a fan of keywords for this sort of thing, since it
implies that you could do this:

def f(x=defer []): ...

dflt = defer []
def f(x=dflt): ...

which is a completely different proposal (eg it would be evaluated
only when you "touch" that, rather than being guaranteed to be
evaluated before the first line of the function body). That's why I
want to adorn the equals sign and nothing else.

Shouldn't the PEP contain a rejected idea section where this could be mentioned?

 

>> 4) If "no" to question 1, is there some other spelling or other small
>> change that WOULD mean you would use it? (Some examples in the PEP.)
>
>
> *Perhaps* if a keyword would be used instead of symbols, I might reconsider.
>
> I find the emphasis of trying to cram too much information in single lines of code to be really a burden. Many years ago, I argued very unsuccessfully for using a 'where:' code block for annotations.  (To this day, I still believe it would make the code much more readable, at the cost of a slight duplication.)  Using what is at first glance a cryptic operator like => for late binding is not helping readability, especially when type annotations are thrown in the mix.
>
> Aside: at the same time, I can see how using => instead of lambda as a potential win in readability, including for beginners.

It's interesting how different people's views go on that sort of
thing. It depends a lot on how much people expect to use something.
Features you use a lot want to have short notations, features you
seldom use are allowed to have longer words.

I rarely use lambda in my own code, and have never written a line of code anywhere that uses a '=>' operator.

 

If Python had a 'function' keyword instead of 'lambda', I would prefer to keep the function keyword instead of adding => as a symbol.  For me, it is not a question of terseness for commonly used features, but one of easing the learning curve.  Starting from zero, I do believe that => would be easier to grasp than learning about lambda as a keyword and the syntactic rules to use with it.  With function as a keyword, I believe that it is the other way around.  No doubt many others will disagree!

André Roberge

[Chris Angelico]

On Thu, Dec 2, 2021 at 12:42 AM David Mertz, Ph.D.
<david...@gmail.com> wrote:
>> 4) If "no" to question 1, is there some other spelling or other small
>> change that WOULD mean you would use it? (Some examples in the PEP.)
>
>
> Yes, the delay/later/defer keyword approach is not confusing, and does not preempt a later feature that would actually be worth having.

Do you mean changing the spelling of the existing proposal, or a
completely different proposal for deferred objects that are actually
objects? Because that is NOT what I mean by a "small change". :)

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/Z2YQZABQWB2GQ5A4XNA2IJB3IN2G365Q/

[Chris Angelico]

On Thu, Dec 2, 2021 at 1:21 AM Paul Moore <p.f....@gmail.com> wrote:
>
> On Wed, 1 Dec 2021 at 06:19, Chris Angelico <ros...@gmail.com> wrote:
> > 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> > defaults, (b) referencing things that might have changed, (c)
> > referencing other arguments, (d) something else?
>
> N/A, except to say that when you enumerate the use cases like this,
> none of them even tempt me to use this feature.

I'm just listing some of the common use-cases, not all of them.

> I think that the only
> thing I might use it for is to make it easier to annotate defaults (as
> f(a: list[int] => []) rather than as f(a: list[int] | None = None).

That's exactly the goal! It's hugely simpler to say "this is always a
list, and defaults to a new empty list" than "this is a list or None,
defaults to None, and hey, if it's None, I'll make a new empty list".

> So I'll revise my answer to (1) and say that I *might* use this, but
> only in a way it wasn't intended to be used in, and mostly because I
> hate how verbose it is to express optional arguments in type
> annotations. (And the fact that the annotation exposes the sentinel
> value, even when you want it to be opaque). I hope I don't succumb and
> do that, though ;-)

That's definitely an intended use-case ("mutable defaults" above);
you're focusing on the fact that it improves the annotations, I'm
focusing on the fact that it improves documentation and introspection,
but it's all the same improvement :)

> > 5) Do you know how to compile CPython from source, and would you be
> > willing to try this out? Please? :)
>
> Sorry, I really don't have time to, in the foreseeable future. If I
> did have time, one thing I would experiment with is how this interacts
> with typing and tools like pyright and mypy (yes, I know type checkers
> would need updating for the new syntax, so that would mostly be a
> thought experiment) - as I say, I'd expect to annotate a function with
> an optional list argument defaulting to an empty list as f(a:
> list[int] => []), which means that __annotations__ needs to
> distinguish between this case and f(a: list[int]) with no default.

No problem, yup.

> Sorry, I don't have any feedback like that. What I can say, though, is
> I'd find it quite hard to express the question, in the sense that I'd
> struggle to explain the difference between early and late bound
> parameters to a non-expert, much less explain why we need both. I'd
> probably just say "it's short for a default of None and a check" which
> doesn't really capture the point...

The point is that it's a shortcut for "omitted" rather than "a default
of None", but there are many ways to explain it, and I only have one
brother who knows enough Python to be able to talk to about this (and
I got feedback from him VERY early).

> > Any and all comments welcomed. I mean, this is python-ideas after
> > all... bikeshedding is what we do best!
>
> I hope this was useful feedback.

It was. Thank you.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/H4VA7EM6YA7WFISGVZ4NJV5N7DQAKYR3/

[David Mertz, Ph.D.]

On Wed, Dec 1, 2021 at 9:24 AM Paul Moore <p.f....@gmail.com> wrote:

I think that the only
thing I might use it for is to make it easier to annotate defaults (as
f(a: list[int] => []) rather than as f(a: list[int] | None = None).

Why not `f(a: Optional[list[int]] = None)`?

I'm not counting characters, but that form seems to express the intention better than either of the others IMHO.

[Chris Angelico]

On Thu, Dec 2, 2021 at 2:24 AM David Mertz, Ph.D. <david...@gmail.com> wrote:
>
> On Wed, Dec 1, 2021 at 9:24 AM Paul Moore <p.f....@gmail.com> wrote:
>>
>> I think that the only
>> thing I might use it for is to make it easier to annotate defaults (as
>> f(a: list[int] => []) rather than as f(a: list[int] | None = None).
>
>
> Why not `f(a: Optional[list[int]] = None)`?
>
> I'm not counting characters, but that form seems to express the intention better than either of the others IMHO.
>

"a: list[int] => []" fully expresses the intention IMO, but yes,
Optional is usually better than "| None". That's just a minor spelling
problem though.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/I6KJR7YPDJOCJDMFVOJXV7BRK6JICBIQ/

[Chris Angelico]

On Thu, Dec 2, 2021 at 1:33 AM André Roberge <andre....@gmail.com> wrote:
> On Wed, Dec 1, 2021 at 7:51 AM Chris Angelico <ros...@gmail.com> wrote:
>>
>> On Wed, Dec 1, 2021 at 10:30 PM André Roberge <andre....@gmail.com> wrote:
>>
>> >> 2) Independently: Is the syntactic distinction between "=" and "=>" a
>> >> cognitive burden?
>> >
>> > Yes.
>> > I really think that using a keyword like defer, or from_calling_scope ;-), would significantly reduce the cognitive burden.
>>
>> Also fair. I'm not a fan of keywords for this sort of thing, since it
>> implies that you could do this:
>>
>> def f(x=defer []): ...
>>
>> dflt = defer []
>> def f(x=dflt): ...
>>
>> which is a completely different proposal (eg it would be evaluated
>> only when you "touch" that, rather than being guaranteed to be
>> evaluated before the first line of the function body). That's why I
>> want to adorn the equals sign and nothing else.
>
>
> Shouldn't the PEP contain a rejected idea section where this could be mentioned?
>

Hmm, maybe. It's such a completely different proposal, but it does get
asked a few times. If someone could actually put it forward as a full
proposal, I'd gladly mention it as an interaction with another PEP.
Otherwise, I'll write up a very brief thing on deferred expressions
and how they're not what this is about.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/7RM5G6T3VHBA7MEWJ53O6N6PB4CDXWPW/

[David Mertz, Ph.D.]

On Wed, Dec 1, 2021 at 10:12 AM Chris Angelico <ros...@gmail.com> wrote:

On Thu, Dec 2, 2021 at 12:42 AM David Mertz, Ph.D.
<david...@gmail.com> wrote:
>> 4) If "no" to question 1, is there some other spelling or other small
>> change that WOULD mean you would use it? (Some examples in the PEP.)
>
> Yes, the delay/later/defer keyword approach is not confusing, and does not preempt a later feature that would actually be worth having.

Do you mean changing the spelling of the existing proposal, or a
completely different proposal for deferred objects that are actually
objects? Because that is NOT what I mean by a "small change". :)

The spelling of the existing proposal.  I.e. if the proposal were:

def fun(things: list[int] = defer []) -> int:

    # ... some implementation

I'd be -0 on the idea rather than -100.

Yes, my change in attitude is largely because I want *some future PEP* to address the more general situation like:

result = defer really_expensive_calculation()

if predicate:

    doubled = result * 2

But I do not think your PEP does (nor even should) include that potential future behavior/syntax.  Such a hypothetical future PEP would have a continuity with the syntax of your feature, albeit DEFINITELY need to address many independent concerns/issues that yours does not create.

However, even if I assume the mythical future PEP never happens, in terms of readability, a WORD is vastly less confusing than a combination of punctuation that has no obvious or natural interpretation like '=>'.  Or rather, I think that spelling is kinda-sorta obvious for the lambda meaning, and the use you want is kinda-sorta similar to a lambda.  So I *do* understand how you get there... but it still seems like much too much line noise for a very minimal need.

[Paul Moore]

On Wed, 1 Dec 2021 at 15:24, David Mertz, Ph.D. <david...@gmail.com> wrote:
>
> On Wed, Dec 1, 2021 at 9:24 AM Paul Moore <p.f....@gmail.com> wrote:
>>
>> I think that the only
>> thing I might use it for is to make it easier to annotate defaults (as
>> f(a: list[int] => []) rather than as f(a: list[int] | None = None).
>
>
> Why not `f(a: Optional[list[int]] = None)`?
>
> I'm not counting characters, but that form seems to express the intention better than either of the others IMHO.

If None were a valid argument, and I was using an opaque sentinel,
Optional doesn't work, and exposing the type of the sentinel is not
what I intend (as it's invalid to explicitly supply the sentinel
value).

Also, Optional[list[int]] doesn't express the intent accurately - the
intended use is that people must supply a list[int] or not supply the
argument *at all*. Optional allows them to supply None as well.

As I say, I don't consider this an intended use case for the feature,
because what I'm actually discussing here is optional arguments and
sentinels, which is a completely different feature. All I'm saying is
that the only case when I can imagine using this feature is for when I
want a genuinely opaque way of behaving differently if the caller
omitted an argument (and using None or a sentinel has been good enough
all these years, so it's not exactly a pressing need).

Let's just go back to the basic point, which is that I can't think of
a realistic case where I'd want to actually use the new feature.

Paul
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/NYHE57O3JVD4GWBIJC57MB5E7PWG6YPR/

[role.python...@jlassocs.com]

Chris Angelico wrote:
> I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/

> Questions, for you all:
> 1) If this feature existed in Python 3.11 exactly as described, would
> you use it?

When needed, but I'm uncomfortable with the syntax.

> 2) Independently: Is the syntactic distinction between "=" and "=>" a
> cognitive burden?

Yes. A lot of arrows with different meanings require the reader of the code to know and remember what each does. Something more distinctive feels better.

I'm way too new to Python to have much say, but anyway, arrows make more sense for other uses. Late binding feels to me like it is best served by something more distinctive.

> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> defaults, (b) referencing things that might have changed, (c)
> referencing other arguments, (d) something else?

I'd certainly use it only when I felt it was absolutely necessary. The readability is not aided by utilising syntax to alter behaviour when the altered behaviour is not required. We only quote, escape, etc., when we need to. (Silly example to illustrate, I know.)

> 4) If "no" to question 1, is there some other spelling or other small
> change that WOULD mean you would use it? (Some examples in the PEP.)

When I first saw this (very few months ago indeed!!), I wondered about something rather clunkier but more explicit, such as, for example --
```def foo(x=__late__['bar']):
def bar():
#something
etc...
```

Ugly though. And I subsequently realised some use cases are not well-served by this. (Particularly when [speculatively] considering some implementation issues.)

The point in any case, is that for the occasions when the mechanism is needed, then a clearly explicit way to indicate something is evaluated late, and a tidy place to put the definition of how it's handled (late), seemed to me to be preferable.

Of the **cited** examples, I find the `@hi=len(a)` spelling most readable.

> 5) Do you know how to compile CPython from source, and would you be
> willing to try this out? Please? :)

Sadly not yet, but time permitting I will one day figure that out, for sure.

> bikeshedding is what we do best!

Should we introduce choice of colour, as we run out of new symbols to use? /jk

JL

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/EJEFUSQRW6CBX5KJTMLDAPIGDRWEV6UZ/

[Ethan Furman]

On 11/30/21 10:16 PM, Chris Angelico wrote:

> *PEP 671: Syntax for late-bound function argument defaults*
>

> Questions, for you all:
>
> 1) If this feature existed in Python 3.11 exactly as described, would
> you use it?

No.

> 2) Independently: Is the syntactic distinction between "=" and "=>" a
> cognitive burden?

Yes.

> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> defaults, (b) referencing things that might have changed, (c)
> referencing other arguments, (d) something else?

a, b, c

> 4) If "no" to question 1, is there some other spelling or other small
> change that WOULD mean you would use it? (Some examples in the PEP.)

Have the token/keyword be at the beginning instead of in the middle.

--
~Ethan~

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/HEC53APMOJA4WR5QTF7Y2VSKWQE62JHU/

[Piotr Duda]

> *PEP 671: Syntax for late-bound function argument defaults*
>
> Questions, for you all:
>
> 1) If this feature existed in Python 3.11 exactly as described, would
> you use it?

Yes

> 2) Independently: Is the syntactic distinction between "=" and "=>" a
> cognitive burden?

No, but it will be cognitive burden with shorthand lambda proposed
syntax, for example
def x(a: (b, c)=>c):
is annotation for a (b, c) or maybe (b, c)=>c

> (It's absolutely valid to say "yes" and "yes", and feel free to say
> which of those pulls is the stronger one.)
>

> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> defaults, (b) referencing things that might have changed, (c)
> referencing other arguments, (d) something else?

mostly (a), sometimes (c)

> 4) If "no" to question 1, is there some other spelling or other small
> change that WOULD mean you would use it? (Some examples in the PEP.)

While I answered yes to question 1, personally I would prefer not
adding new syntax, but switching completly to late defaults (requiring
future import for some next versions)

> 5) Do you know how to compile CPython from source, and would you be
> willing to try this out? Please? :)

Don't have enough time.

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/EAUYTHGRRA337IEM4AZBVBFPIW2SLSPT/

[Chris Angelico]

On Thu, Dec 2, 2021 at 2:40 AM David Mertz, Ph.D. <david...@gmail.com> wrote:
>
> On Wed, Dec 1, 2021 at 10:12 AM Chris Angelico <ros...@gmail.com> wrote:
>>
>> On Thu, Dec 2, 2021 at 12:42 AM David Mertz, Ph.D.
>> <david...@gmail.com> wrote:
>> >> 4) If "no" to question 1, is there some other spelling or other small
>> >> change that WOULD mean you would use it? (Some examples in the PEP.)
>> >
>> > Yes, the delay/later/defer keyword approach is not confusing, and does not preempt a later feature that would actually be worth having.
>>
>> Do you mean changing the spelling of the existing proposal, or a
>> completely different proposal for deferred objects that are actually
>> objects? Because that is NOT what I mean by a "small change". :)
>
>
> The spelling of the existing proposal. I.e. if the proposal were:
>
> def fun(things: list[int] = defer []) -> int:
> # ... some implementation
>
>
> I'd be -0 on the idea rather than -100.

Okay. If it's JUST the spelling, then yes, I'll take that into
consideration (though I am still against keywords myself).

> Yes, my change in attitude is largely because I want *some future PEP* to address the more general situation like:
>
> result = defer really_expensive_calculation()
> if predicate:
> doubled = result * 2
>
>
> But I do not think your PEP does (nor even should) include that potential future behavior/syntax. Such a hypothetical future PEP would have a continuity with the syntax of your feature, albeit DEFINITELY need to address many independent concerns/issues that yours does not create.
>
> However, even if I assume the mythical future PEP never happens, in terms of readability, a WORD is vastly less confusing than a combination of punctuation that has no obvious or natural interpretation like '=>'. Or rather, I think that spelling is kinda-sorta obvious for the lambda meaning, and the use you want is kinda-sorta similar to a lambda. So I *do* understand how you get there... but it still seems like much too much line noise for a very minimal need.
>

The trouble is that this actually would be incompatible. If you can
defer an expensive calculation and have some "placeholder" value in
the variable 'result', then logically, you should be able to have that
placeholder as a function default argument, which would be an
early-bound default of the placeholder. That is quite different in
behaviour from a late-bound default, so if I were to use the word
"defer" for late-bound defaults, it would actually prevent the more
general proposal.

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/VM2DEOHXYCY57VPPWBUCW7G6NJOSDVAN/

[Barry Scott]

On 1 Dec 2021, at 06:16, Chris Angelico <ros...@gmail.com> wrote:

I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/

with some additional information about the reference implementation,
and some clarifications elsewhere.

*PEP 671: Syntax for late-bound function argument defaults*

Questions, for you all:

1) If this feature existed in Python 3.11 exactly as described, would
you use it?

no because of name=>

2) Independently: Is the syntactic distinction between "=" and "=>" a
cognitive burden?

yes.

(It's absolutely valid to say "yes" and "yes", and feel free to say
which of those pulls is the stronger one.)

3) If "yes" to question 1, would you use it for any/all of (a) mutable
defaults, (b) referencing things that might have changed, (c)
referencing other arguments, (d) something else?

yes (a)

What does (b) mean? example please.

yes (c)

4) If "no" to question 1, is there some other spelling or other small
change that WOULD mean you would use it? (Some examples in the PEP.)

Use the @name to avoid the confusing with the set of = things.

5) Do you know how to compile CPython from source, and would you be
willing to try this out? Please? :)

no promises, if I get spare time I'll give it a go,

should be easy to hack the Fedora python RPM to build your

version.

I'd love to hear, also, from anyone's friends/family who know a bit of
Python but haven't been involved in this discussion. If late-bound
defaults "just make sense" to people, that would be highly
informative.

Any and all comments welcomed. I mean, this is python-ideas after

all... bikeshedding is what we do best!

The reference implementation currently has some test failures, which
I'm looking into. I'm probably going to make this my personal default
Python interpreter for a while, to see how things go.

Barry

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/UVOQEK7IRFSCBOH734T5GFJOEJXFCR6A/

[Chris Angelico]

On Thu, Dec 2, 2021 at 4:40 AM Barry Scott <ba...@barrys-emacs.org> wrote:
>
> On 1 Dec 2021, at 06:16, Chris Angelico <ros...@gmail.com> wrote:
> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> defaults, (b) referencing things that might have changed, (c)
> referencing other arguments, (d) something else?
>
>
> yes (a)
> What does (b) mean? example please.
> yes (c)
>

global_default = 500
def do_thing(timeout=>global_default): ...

If the global_default timeout changes between function definition and
call, omitting timeout will use the updated global.

Similarly, you could say "file=>sys.stdout" and if code elsewhere
changes sys.stdout, you'll use that.

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/KVLO3CEXBJUKBUJPJZIJM54U6S5PIFKM/

[Barry Scott]

> On 1 Dec 2021, at 17:59, Chris Angelico <ros...@gmail.com> wrote:
>
> On Thu, Dec 2, 2021 at 4:40 AM Barry Scott <ba...@barrys-emacs.org> wrote:
>>
>> On 1 Dec 2021, at 06:16, Chris Angelico <ros...@gmail.com> wrote:
>> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
>> defaults, (b) referencing things that might have changed, (c)
>> referencing other arguments, (d) something else?
>>
>>
>> yes (a)
>> What does (b) mean? example please.
>> yes (c)
>>
>
> global_default = 500
> def do_thing(timeout=>global_default): ...

>
> If the global_default timeout changes between function definition and
> call, omitting timeout will use the updated global.
>
> Similarly, you could say "file=>sys.stdout" and if code elsewhere
> changes sys.stdout, you'll use that.

On a case-by-case basis I might still put defaulting into the body
of the function if that made the intent clearer.

I could see me using @file=sys.stdout.

Barry

>
> ChrisA
> _______________________________________________
> Python-ideas mailing list -- python...@python.org
> To unsubscribe send an email to python-id...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at https://mail.python.org/archives/list/python...@python.org/message/KVLO3CEXBJUKBUJPJZIJM54U6S5PIFKM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/IZIGDOR2NE2N774SJ4PZ4L7IOWVTUDKF/

[Chris Angelico]

On Thu, Dec 2, 2021 at 8:50 AM Barry Scott <ba...@barrys-emacs.org> wrote:
>
>
>
> > On 1 Dec 2021, at 17:59, Chris Angelico <ros...@gmail.com> wrote:
> >
> > On Thu, Dec 2, 2021 at 4:40 AM Barry Scott <ba...@barrys-emacs.org> wrote:
> >>
> >> On 1 Dec 2021, at 06:16, Chris Angelico <ros...@gmail.com> wrote:
> >> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> >> defaults, (b) referencing things that might have changed, (c)
> >> referencing other arguments, (d) something else?
> >>
> >>
> >> yes (a)
> >> What does (b) mean? example please.
> >> yes (c)
> >>
> >
> > global_default = 500
> > def do_thing(timeout=>global_default): ...
>
> >
> > If the global_default timeout changes between function definition and
> > call, omitting timeout will use the updated global.
> >
> > Similarly, you could say "file=>sys.stdout" and if code elsewhere
> > changes sys.stdout, you'll use that.
>
> On a case-by-case basis I might still put defaulting into the body
> of the function if that made the intent clearer.
>
> I could see me using @file=sys.stdout.
>

That's a simplified version, but you might have the same default shown
as get_default_timeout() or Defaults.timeout or something like that.
The point is that it might be a simple integer, but it could change at
any time, and the default should always be "whatever this name refers
to".

In any case, it's just one of many use-cases. I was curious what
people would use and what they wouldn't.

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/24RHPWAN6TF2LE74QM7GCRHW6EUUSZWH/

[Greg Ewing]

On 2/12/21 4:40 am, Paul Moore wrote:
> the
> intended use is that people must supply a list[int] or not supply the
> argument *at all*.

I don't think this is a style of API that we should be encouraging
people to create, because it results in things that are very
awkward to wrap.

--
Greg

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/TUQPAOSSNHGS5QZPWE7OQF5XF2IIUVBV/

[Jeremiah Vivian]

To be honest, I like the `@param=value` syntax. It is sort of easier to read than `param=>value`, though I do not have problems with distinguishing the arrow `=>` from other things.

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/AG2JBB5SYNZNW6P7VYHU3LBTDHJU3TIV/

[Steven D'Aprano]

On Wed, Dec 01, 2021 at 12:26:33PM +0000, Matt del Valle wrote:

> If anything I think
> the symmetry between the proposed '=>' syntax and the arrow syntax for
> lambdas in other languages (potentially even in python in the future)
> reduces the cognitive burden significantly, given the there is an
> equivalent symmetry with their semantics (in both cases the code is being
> evaluated later when something is called).

There is not as much symmetry as you might think between a hypothetical
lambda arrow and the proposed late-bound default arrow.

arg => arg + 1 # lambda
arg=>expr # late-bound default

The first case is (or could be some day...) an actual expression that
returns a function object, which we will explicitly call at some point.
Or at least pass the function to another function, which will call it.

But the late bound default is not an expression, it is a declaration. It
declares the default value used for arg. We don't have to call anything
to get access to the default value. It just shows up when we access the
parameter without providing an argument for it.

We certainly don't need to call arg explicitly to evaluate the default.

It may be that behind the scenes the default expression is stored as a
callable function which the interpreter calls. (I believe that list
comprehensions do something similar.) But that's an implementation
detail that can change: it might just as well store the source code as a
string, and pass it to eval().

Or use some other mechanism that I'm not clever enough to think of, so I
shall just call "deepest black magic".

There is no *neccessity* for the late-bound default to be a hidden
function, and it is certainly not part of the semantics of late-bound
defaults. Just the implementation.

If you disagree, and still think that the symmetry is powerful enough to
use the same syntax for both lambdas and default arguments, well, how
about if we *literally* do that?

def function(spam=expression, # regular default
lambda eggs: expression, # late-bound default
)

Heh, perhaps the symmetry is not that strong after all :-)



--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/J6KVJX2HEXRNVPEZAEBEFXGW7SLSKWNY/

[Chris Angelico]

On Thu, Dec 2, 2021 at 1:30 PM Steven D'Aprano <st...@pearwood.info> wrote:
>
> On Wed, Dec 01, 2021 at 12:26:33PM +0000, Matt del Valle wrote:
>
> > If anything I think
> > the symmetry between the proposed '=>' syntax and the arrow syntax for
> > lambdas in other languages (potentially even in python in the future)
> > reduces the cognitive burden significantly, given the there is an
> > equivalent symmetry with their semantics (in both cases the code is being
> > evaluated later when something is called).
>

> It may be that behind the scenes the default expression is stored as a
> callable function which the interpreter calls. (I believe that list
> comprehensions do something similar.) But that's an implementation
> detail that can change: it might just as well store the source code as a
> string, and pass it to eval().

In my reference implementation, there is no object that stores it;
it's simply part of the function. A good parallel is the if/else
expression:

x = float("inf") if z == 0 else y/z

Is there an object that represents the if/else expression (or the
deferred "y/z" part)? No, although it could be implemented that way if
you chose:

x = iff(z == 0, lambda: y/z, lambda: float("inf"))

But just because you CAN use a function to simulate this behaviour,
that doesn't mean that it's inherently a function.

So I agree with Steve that the parallel is quite weak. That said,
though, I still like the arrow notation, not because of any parallel
with a lambda function, but because of the parallel with assignment.

(And hey. You're allowed to think of things in any way you like. I'm
not forcing you to interpret everything using the same justifications
I do.)

> Or use some other mechanism that I'm not clever enough to think of, so I
> shall just call "deepest black magic".

Remind me some time to use some slightly shallower black magic.

> There is no *neccessity* for the late-bound default to be a hidden
> function, and it is certainly not part of the semantics of late-bound
> defaults. Just the implementation.
>
> If you disagree, and still think that the symmetry is powerful enough to
> use the same syntax for both lambdas and default arguments, well, how
> about if we *literally* do that?
>
> def function(spam=expression, # regular default
> lambda eggs: expression, # late-bound default
> )
>
> Heh, perhaps the symmetry is not that strong after all :-)

That..... is hilariously confusing. I like it. Just not EVER in the language :)

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/AQWFM7KYVMLBD6FH7XDYLMQVUBZOJGNW/

[Steven D'Aprano]

On Wed, Dec 01, 2021 at 05:16:34PM +1100, Chris Angelico wrote:

> 1) If this feature existed in Python 3.11 exactly as described, would
> you use it?

Yes I would, but probably not as often as counting cases of the "if
param is None: ..." idiom might lead you to expect.

The problem is, as Neil also pointed out, that it becomes tricky to
explicitly ask for the default behaviour except by leaving the argument
out altogether. Not impossible, as Chris mentions elsewhere, you can
mess about with `*args` or `**kwargs`, but it is decidedly less
convenient, more verbose, and likely to have a performance hit.

So if I were messing about in the interactive interpreter, I would
totally use this for the convenience:

def func(L=>[]): ...

but if I were writing a library, I reckon that probably at least half
the time I'll stick to the old idiom so I can document the parameter:

"If param is missing **or None**, the default if blah..."

I reject Chris' characterisation of this as a hack. There are function
parameters where None will *never* in any conceivable circumstances
become a valid argument value, and it is safe to use it as a sentinel.

For example, I have a function that takes a `collation=None` parameter.
The collation is a sequence of characters, usually a string. Using None
to indicate "use the default collation" will never conflict with some
possible future use of "use None as the actual collation" because None
is not a sequence of characters.

In this specific case, my function's current signature is:

def itoa(n, base,
collation=None,
plus='',
minus='-',
width=0,
):

I could re-write it as:

def itoa(n, base,
collation=>_default_collation(base),
plus='',
minus='-',
width=0,
):

but that would lose the ability to explicitly say "use the default
collation" by passing None. So I think that, on balance, I would
likely stick to the existing idiom for this function.

On the other hand, if None were a valid value, so the signature used a
private and undocumented sentinel:

collation=_MISSING

(and I've used that many times in other functions) then I would be
likely to swap to this feature and drop the private sentinel.

> 2) Independently: Is the syntactic distinction between "=" and "=>" a
> cognitive burden?

I think you know my opinion on the `=>` syntax :-)

I shall not belabour it further here. I reserve the right to belabour it
later on :-)

> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> defaults, (b) referencing things that might have changed, (c)
> referencing other arguments, (d) something else?

Any of the above.

> 4) If "no" to question 1, is there some other spelling or other small
> change that WOULD mean you would use it? (Some examples in the PEP.)

There's lots of Python features that I use even though I don't like the
spelling. Not a day goes by that I don't wish we spelled the base class
of the object hierarchy "Object", so it were easier to distinguish
between Object the base class and object as a generic term for any
object.

If the Steering Council loves your `=>` syntax then I would disagree
with their decision but still use it.


--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/G3V47ST6INLI7Y3C5RZSWVXFLNUTELQT/

[David Mertz, Ph.D.]

On Wed, Dec 1, 2021, 10:38 PM Steven D'Aprano 

"If param is missing **or None**, the default if blah..."
I reject Chris' characterisation of this as a hack. There are function parameters where None will *never* in any conceivable circumstances become a valid argument value, and it is safe to use it as a sentinel.

In particular, the cases where None will never, in any conceivable circumstances, be a non-sentinel value are at least 98% of all the functions (that have named parameters) I've ever written in Python.

All of this discussion of a syntax change is for, at most, 2% of functions that need a different sentinel.

[Abe Dillon]

I like the => syntax and would use it probably 80% of the time for mutable defaults.

I don't think it causes cognitive load.

I especially disagree with the argument that it looks like arbitrary symbols and adds more 'line noise' than a keyword. My eye picks it up as a distinct arrow rather than a random assembly of punctuation. Maybe it's not as hard as people claim to visually parse strings of punctuation that make little pictures ¯\_(ツ)_/¯

I also disagree that it makes Python harder to teach or learn. You *have to* teach students about early binding anyway. That's the hard part. Explaining how parameter binding works so that they can avoid the pit traps is the hard part. Then you either show them a verbose and clumsy work around of checking for a sentinel or you show them a cute little arrow.

I think people are missing the overall point of doing in the function header what belongs in the function header so that it doesn't spill into the body.

My favorite alternative is ?= if people think  =>  and -> are getting overly loaded. What I really don't like is @param=[] because it puts the emphasis on the parameter name rather than the act of binding. Not only does it make it look like @param is a special kind of variable, it also mimics the *args and **kwargs syntax which makes them seem related.

[abed...@gmail.com]

Steven D'Aprano

""If param is missing **or None**, the default if blah..."

I reject Chris' characterisation of this as a hack. There are function
parameters where None will *never* in any conceivable circumstances
become a valid argument value, and it is safe to use it as a sentinel."

Yes, we know *why* the hack works. We're all familiar with it. That doesn't mean it's not a hack.

The bottom line is:
you *don't actually* want the parameter to default to the value of a sentinel.

you *have* to use that hack because you can't express what you want the default to actually be.

You're doing something misleading to work around a shortcoming of the language.
That's a hack.

You have to write something that you don't actually intend.

[abed...@gmail.com]

2% of functions is a lot of functions. We're talking about a language that's been around 30 years. The feature set is pretty mature. If it were lacking features that would improve a much larger percent of code for so long, I don't think it would be as popular. It's not like the next PIP is going to be as big as the for-loop.

[Abdulla Al Kathiri]

Exactly like you. I can read => to mean late-bound or supposedly lambda function and I can distinguish it from the callable typing ->  but "@param=value” is definitely better in my eyes. => won’t prevent me from using it for default mutable objects.

[Brendan Barnwell]

On 2021-11-30 22:16, Chris Angelico wrote:
> I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
> with some additional information about the reference implementation,
> and some clarifications elsewhere.
>
> *PEP 671: Syntax for late-bound function argument defaults*
>
> Questions, for you all:
>

> 1) If this feature existed in Python 3.11 exactly as described, would
> you use it?

I hope not. :-)

Realistically I might wind up using it at some point way down the line.
I wouldn't start using it immediately. I still almost never use the
walrus and only occasionally use f-strings.

You didn't ask how people would feel about READING this rather than
writing it, but what I would do is get really annoyed at seeing it in
code as people start to use it and confuse me and others.

> 2) Independently: Is the syntactic distinction between "=" and "=>" a
> cognitive burden?
>

> (It's absolutely valid to say "yes" and "yes", and feel free to say
> which of those pulls is the stronger one.)

Yes, it is yet another reason not to do this.

> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> defaults, (b) referencing things that might have changed, (c)
> referencing other arguments, (d) something else?

N/A.

> 4) If "no" to question 1, is there some other spelling or other small
> change that WOULD mean you would use it? (Some examples in the PEP.)

No. As I mentioned in the earlier thread, I don't support any proposal
in which an argument can "have a default" but that default is not a
first-class Python object of some sort.

> 5) Do you know how to compile CPython from source, and would you be
> willing to try this out? Please? :)

No, I don't.

I know I said this before, but I really hope this change is not
adopted. It is to me a classic example of adding significant complexity
to the language and reducing readability for only a very small benefit
in expressiveness.

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."
--author unknown

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/NKZNWRTCGB53MNLMWJLMBYQAVAIRZWKO/

[role.python...@jlassocs.com]

Brendan Barnwell wrote:
> > No. As I mentioned in the earlier thread, I don't support any proposal
> in which an argument can "have a default" but that default is not a
> first-class Python object of some sort.

What if a default is a function?

I was inspired by learning Django and saw in models that fields can have a default which is either a regular (early-bound) default such as a first-class Python object as one would expect, *or* a function -- which will be called 'later' when needed.

That prompted me to contemplate a syntax for late-bound defaults, albeit a bit clunky, but I did think it suited a special-case requirement met by late-bound defaults. I still think that littering function arguments throughout all code with large numbers of arrows would make things less readable. Special requirements need special treatment, I'm thinking.

The problem is passing arguments to such a function without it looking like it's being called at definition time.

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/PJNVHV3UY5L67GDEZWGXMRPIL6SN4563/

[Chris Angelico]

On Thu, Dec 2, 2021 at 6:12 PM <role.python...@jlassocs.com> wrote:
>
> Brendan Barnwell wrote:
> > > No. As I mentioned in the earlier thread, I don't support any proposal
> > in which an argument can "have a default" but that default is not a
> > first-class Python object of some sort.
>
> What if a default is a function?
>
> I was inspired by learning Django and saw in models that fields can have a default which is either a regular (early-bound) default such as a first-class Python object as one would expect, *or* a function -- which will be called 'later' when needed.
>
> That prompted me to contemplate a syntax for late-bound defaults, albeit a bit clunky, but I did think it suited a special-case requirement met by late-bound defaults. I still think that littering function arguments throughout all code with large numbers of arrows would make things less readable. Special requirements need special treatment, I'm thinking.
>
> The problem is passing arguments to such a function without it looking like it's being called at definition time.
>

Also has the same problem of other deferreds, which is: when exactly
is it evaluated?

def func(stuff, n=>len(stuff)):
stuff.append("spam")
print(n)

func(["spam", "ham"])

What will this print? If it's a function default, it MUST print 2,
since len(stuff) is 2 as the function starts. But if it's a deferred
object of some sort, then it should probably print 3, since len(stuff)
is 3 at the time that n gets printed. Which is it to be?

That's why PEP 671 is *not* about generic deferred evaluation. It is
specifically about function default arguments, and guarantees that
they will be evaluated prior to the first line of code in the function
body.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/7IQQ34WYQOJGSBMLWINYV4NSOAR6ZTOH/

[Brendan Barnwell]

On 2021-12-01 18:35, Chris Angelico wrote:
> In my reference implementation, there is no object that stores it;
> it's simply part of the function. A good parallel is the if/else
> expression:
>
> x = float("inf") if z == 0 else y/z
>
> Is there an object that represents the if/else expression (or the
> deferred "y/z" part)? No, although it could be implemented that way if
> you chose:

This is not a good parallel. There is nothing deferred there. The
entire line is evaluated when it is encountered and you get a result and
no part of the if/else expression can ever impact anything else again
unless, via some external control flow, execution returns and
re-executes the entire line. That is not comparable to a function
default, which is STORED and evaluated later independently of the
context in which it was originally written (i.e., the function default
is re-executed but the function definition itself is not re-executed).

The ternary expression vanishes without a trace by the next line,
leaving only its evaluated result. There would be no use in being able
to access some part of it, since the whole (i.e., the ternary
expression) is completely finished by the time you would be able to
access it. This is not the case with a function definition. The
function definition leaves behind a function object, and that function
object needs to "know" about the late-bound default as an independent
entity so that it can be evaluated later. It is bad for the function to
store that late-bound default only in some private format for its
exclusive future use without providing any means for other code to
access it as a first-class value.

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."
--author unknown

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/43TVQCJ62PJ2ZRCYFC7M467CYJOXA6FP/

[Chris Angelico]

That's exactly why it's such a close parallel. The late-evaluated
default is just code, nothing else. It's not "stored" in any way - it
is evaluated as part of the function beginning execution.

There is no "first class object" for a late-evaluated default any more
than there is one for the "y/z" part of the ternary.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/K6U2NPK6SBEG4NSSFJOMVRX5WQKQH7AX/

[Brendan Barnwell]

On 2021-12-01 23:36, Chris Angelico wrote:
> That's exactly why it's such a close parallel. The late-evaluated
> default is just code, nothing else. It's not "stored" in any way - it
> is evaluated as part of the function beginning execution.

But it IS stored! There is no way for it to be evaluated without it
being stored!

I know we're talking past each other here but it is quite obvious that
something has to be stored if it is going to be evaluated later. You
can say that it is "just code" but that doesn't change the fact that
that code has to be stored. You can say that it is just prepended to
the function body but that's still storing it. That is still not
parallel to a ternary operator in which no part of the expression is
EVER re-executed unless control flow causes execution to return to that
same source code line and re-execute it as a whole.

Actually this raises a question that maybe was answered in the earlier
thread but if so I forgot: if a function has a late-bound default, will
the code to evaluate it be stored as part of the function's code object?

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."
--author unknown

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/O6RF5ZXTG24FFTRHWTIYHKBVOWZIBPSO/

[Paul Moore]

On Wed, 1 Dec 2021 at 22:27, Greg Ewing <greg....@canterbury.ac.nz> wrote:
>
> On 2/12/21 4:40 am, Paul Moore wrote:
> > the
> > intended use is that people must supply a list[int] or not supply the
> > argument *at all*.
>
> I don't think this is a style of API that we should be encouraging
> people to create, because it results in things that are very
> awkward to wrap.

Hmm, interesting point, I agree with you. It's particularly telling
that I got sucked into designing that sort of API, even though I know
it's got this problem. I guess that counts as an argument against the
late bound defaults proposal - or maybe even two:

1. It's hard (if not impossible) to wrap functions that use late-bound defaults.
2. The feature encourages people to write such unwrappable functions
when an alternative formulation that is wrappable is just as good.

(That may actually only be one point - obviously a feature encourages
people to use it, and any feature can be over-used. But the point
about wrappability stands).

Paul

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/HMQVEXZSUWGJSXNGGSDKVBDWNF5PYMUX/

[Chris Angelico]

On Thu, Dec 2, 2021 at 6:59 PM Brendan Barnwell <bren...@brenbarn.net> wrote:
>
> On 2021-12-01 23:36, Chris Angelico wrote:
> > That's exactly why it's such a close parallel. The late-evaluated
> > default is just code, nothing else. It's not "stored" in any way - it
> > is evaluated as part of the function beginning execution.
>
> But it IS stored! There is no way for it to be evaluated without it
> being stored!
>
> I know we're talking past each other here but it is quite obvious that
> something has to be stored if it is going to be evaluated later. You
> can say that it is "just code" but that doesn't change the fact that
> that code has to be stored. You can say that it is just prepended to
> the function body but that's still storing it. That is still not
> parallel to a ternary operator in which no part of the expression is
> EVER re-executed unless control flow causes execution to return to that
> same source code line and re-execute it as a whole.

I'm not sure I understand you here. How is the late-bound default
"stored" when one side of a ternary is "not stored"?

> Actually this raises a question that maybe was answered in the earlier
> thread but if so I forgot: if a function has a late-bound default, will
> the code to evaluate it be stored as part of the function's code object?
>

Yes. To be precise, it is part of the code object's co_code attribute
- the bytecode (or wordcode if you prefer) of the function.

Here's how a ternary if looks:

>>> def f(n):
... return 0 if n == 0 else 42/n
...
>>> dis.dis(f)
2 0 LOAD_FAST 0 (n)
2 LOAD_CONST 1 (0)
4 COMPARE_OP 2 (==)
6 POP_JUMP_IF_FALSE 6 (to 12)
8 LOAD_CONST 1 (0)
10 RETURN_VALUE
>> 12 LOAD_CONST 2 (42)
14 LOAD_FAST 0 (n)
16 BINARY_TRUE_DIVIDE
18 RETURN_VALUE

The "42/n" part is stored in f.__code__.co_code as the part that says
"LOAD_CONST 42, LOAD_FAST n, BINARY_TRUE_DIVIDE". It's not an object.
It's just code - three instructions.

Here's how (in the reference implementation - everything is subject to
change) a late-bound default looks:

>>> def f(x=>[]): print(x)
...
>>> dis.dis(f)
1 0 QUERY_FAST 0 (x)
2 POP_JUMP_IF_TRUE 4 (to 8)
4 BUILD_LIST 0
6 STORE_FAST 0 (x)
>> 8 LOAD_GLOBAL 0 (print)
10 LOAD_FAST 0 (x)
12 CALL_FUNCTION 1
14 POP_TOP
16 LOAD_CONST 0 (None)
18 RETURN_VALUE

The "=>[]" part is stored in f.__code__.co_code as the part that says
"QUERY_FAST x, and if false, BUILD_LIST, STORE_FAST x". It's not an
object. It's four instructions in the bytecode.

In both cases, no part of the expression is ever re-executed. I'm not
understanding the distinction here. Can you explain further please?

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/7QV4GCAMP65H77W7RM732IHNOCPHOQSW/

[Paul Moore]

On Thu, 2 Dec 2021 at 08:29, Paul Moore <p.f....@gmail.com> wrote:
>
> On Wed, 1 Dec 2021 at 22:27, Greg Ewing <greg....@canterbury.ac.nz> wrote:
> >
> > On 2/12/21 4:40 am, Paul Moore wrote:
> > > the
> > > intended use is that people must supply a list[int] or not supply the
> > > argument *at all*.
> >
> > I don't think this is a style of API that we should be encouraging
> > people to create, because it results in things that are very
> > awkward to wrap.
>
> Hmm, interesting point, I agree with you. It's particularly telling
> that I got sucked into designing that sort of API, even though I know
> it's got this problem. I guess that counts as an argument against the
> late bound defaults proposal - or maybe even two:
>
> 1. It's hard (if not impossible) to wrap functions that use late-bound defaults.
> 2. The feature encourages people to write such unwrappable functions
> when an alternative formulation that is wrappable is just as good.
>
> (That may actually only be one point - obviously a feature encourages
> people to use it, and any feature can be over-used. But the point
> about wrappability stands).

Actually, Chris - does functools.wraps work properly in your
implementation when wrapping functions with late-bound defaults?

>>> def dec(f):
... @wraps(f)
... def inner(*args, **kw):
... print("Calling")
... return f(*args, **kw)
... return inner
...
>>> @dec
... def g(a => []):
... return len(a)

Paul
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/M5OHAWF44FRD6OL5F65IDDNKBFL36RUW/

[Chris Angelico]

On Thu, Dec 2, 2021 at 7:31 PM Paul Moore <p.f....@gmail.com> wrote:
>
> On Wed, 1 Dec 2021 at 22:27, Greg Ewing <greg....@canterbury.ac.nz> wrote:
> >
> > On 2/12/21 4:40 am, Paul Moore wrote:
> > > the
> > > intended use is that people must supply a list[int] or not supply the
> > > argument *at all*.
> >
> > I don't think this is a style of API that we should be encouraging
> > people to create, because it results in things that are very
> > awkward to wrap.
>
> Hmm, interesting point, I agree with you. It's particularly telling
> that I got sucked into designing that sort of API, even though I know
> it's got this problem. I guess that counts as an argument against the
> late bound defaults proposal - or maybe even two:
>
> 1. It's hard (if not impossible) to wrap functions that use late-bound defaults.
> 2. The feature encourages people to write such unwrappable functions
> when an alternative formulation that is wrappable is just as good.
>
> (That may actually only be one point - obviously a feature encourages
> people to use it, and any feature can be over-used. But the point
> about wrappability stands).
>

Wrappability when using an arbitrary sentinel looks like this:

_SENTINEL = object()
def f(value=_SENTINEL):
if value is _SENTINEL: value = ...

# in another module
def wrap_f():
othermodule.f(othermodule._SENTINEL)

Or:

def wrap_f():
othermodule.f(othermodule.f.__defaults__[0])

I'm not sure either of these is any better than the alternatives.
You're reaching into a module to access its internal implementation
details. Is that really better than messing with *args?

def wrap_f():
args = [42] or [] # pass or don't pass?
othermodule.f(*args)

It only looks easy when None is used, and even then, it's largely an
implementation detail.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/LF4Q4TPOBQYK5VE2UPYFEFFMGD32NYAG/

[Chris Angelico]

On Thu, Dec 2, 2021 at 7:36 PM Paul Moore <p.f....@gmail.com> wrote:
>
> Actually, Chris - does functools.wraps work properly in your
> implementation when wrapping functions with late-bound defaults?
>
> >>> def dec(f):
> ... @wraps(f)
> ... def inner(*args, **kw):
> ... print("Calling")
> ... return f(*args, **kw)
> ... return inner
> ...
> >>> @dec
> ... def g(a => []):
> ... return len(a)
>

Yes, it does. There are two things happening here which, from a
technical standpoint, are actually orthogonal; the function *call* is
simply using *a,**kw notation, so it's passing along all the
positional and keyword parameters untouched; but the function
*documentation* just says "hey look over there for the real
signature".

>>> g([1,2,3])
Calling
3
>>> g()
Calling
0

(In your example there's no proof that it's late-bound, but it is.)

>>> help(g)
Help on function g in module __main__:

g(a=>[])
>>> g.__wrapped__
<function g at 0x7fb5581efa00>
>>> g
<function g at 0x7fb5581efab0>

When you do "inner = wraps(f)(inner)", what happens is that the
function's name and qualname get updated, and then __wrapped__ gets
added as a pointer saying "hey, assume that I have the signature of
that guy over there". There's unfortunately no way to merge signatures
(you can't do something like "def f(*a, timeout=500, **kw):" and then
use the timeout parameter in your wrapper and pass the rest on -
help(f) will just show the base function's signature), and nothing is
actually aware of the underlying details.

But on the plus side, this DOES work correctly.

Documentation for late-bound defaults is done by a (compile-time)
string snapshot of the AST that defined the default, so it's about as
accurate as for early-bound defaults. One thing I'm not 100% happy
with in the reference implementation is that the string for help() is
stored on the function object, but the behaviour of the late-bound
default is inherent to the code object. I'm not sure of a way around
that, but it also doesn't seem terribly problematic in practice.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/TRDJLI24Z4BM3UCBVSV6E4FASDLT6CQY/

[Steven D'Aprano]

On Wed, Dec 01, 2021 at 10:50:38PM -0800, Brendan Barnwell wrote:

> >4) If "no" to question 1, is there some other spelling or other small
> >change that WOULD mean you would use it? (Some examples in the PEP.)
>
> No. As I mentioned in the earlier thread, I don't support any
> proposal in which an argument can "have a default" but that default is not
> a first-class Python object of some sort.

I don't understand this criticism.

Of course the default value will be a first-class Python object of some
sort. *Every* value in Python is a first-class object. There are no
machine values or unboxed values, and this proposal will not change
that.

All that this proposal changes is *when* and *how often* the default
will be evaluated, not the nature of the value.

Status quo: default values are evaluated once, when the def statement
is executed.

With optional late-binding: default values are evaluated as often as
they are needed, when the function is called. But the value will still
be an object.

I suppose that there will be one other change, relating to introspection
of the function. You will no longer be able to inspect the function and
see the default values as constants in a cache:

>>> (lambda x=1.25: None).__defaults__
(1.25,)

Depending on the implementation, you *might* be able to inspect the
function and see the default expression as some sort of callable
function, or evaluatable code object. (That would be nice.) Or even as a
plain old string. All of which are first-class objects. Or it might be
that the default expression will be compiled into the body of the
function, where is it effectively invisible. So I guess that's a third
change: when, how often, and the difference to introspection.



--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/V6YOQ7RCCNIWXM6KGLC767CHBKZQPES5/

[Steven D'Aprano]

On Thu, Dec 02, 2021 at 08:29:58AM +0000, Paul Moore wrote:

> 1. It's hard (if not impossible) to wrap functions that use late-bound
> defaults.

I don't understand that argument.

We can implement late-bound defaults right now, using the usual sentinel
jiggery-pokery.

def func(spam, eggs, cheese=None, aardvark=42):
if cheese is None: ... # You know the drill.

We can wrap functions like this. You don't generally even care what the
sentinel is. A typical wrapper function looks something like this:

@functools.wraps(func)
def inner(*args, **kwargs):
# wrapper implementation
func(*args, **kwargs)

with appropriate pre-processing or post-processing as needed.

Why would argument unpacking to call the wrapped function stop working?


--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/AHPM6SKTBJZLQBLUYULFPS2K77NJ2LAR/

[Chris Angelico]

On Thu, Dec 2, 2021 at 8:40 PM Steven D'Aprano <st...@pearwood.info> wrote:
> Depending on the implementation, you *might* be able to inspect the
> function and see the default expression as some sort of callable
> function, or evaluatable code object. (That would be nice.)

Unfortunately not, since the default expression could refer to other
parameters, or closure variables, or anything else from the context of
the called function. So you won't be able to externally evaluate it.

> Or even as a plain old string. All of which are first-class objects.

For documentation purposes, it is indeed available as a plain old
string. In colloquial terms, it is the source code for the expression,
although technically it's reconstructed from the AST. ( This is
approximately as accurate as saying that the repr of an object is its
source code. Possibly more accurate, actually.)

> Or it might be
> that the default expression will be compiled into the body of the
> function, where is it effectively invisible. So I guess that's a third
> change: when, how often, and the difference to introspection.

Among our changes are when, how often, the difference to
introspection, and the ability to externally manipulate the defaults.
And nice red uniforms.

Although the ability to manipulate them could be considered part of
introspection, not sure.

I'm still unsure whether this is a cool feature or an utter abomination:

>>> def f(x=...):
... try: print("You passed x as", x)
... except UnboundLocalError: print("You didn't pass x")
...
>>> f.__defaults_extra__ = ("n/a",)
>>> f(42)
You passed x as 42
>>> f()
You didn't pass x
>>>

(It's an implementation detail and not part of the specification, but
if CPython adopts this behaviour, it will become de facto part of the
expected behaviour, and someone somewhere will use this deliberately.)

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/F42ABHBQG2BN3DOUQGAHCBNGPS52JRVY/

[Rob Cliffe via Python-ideas]

On 01/12/2021 10:05, Steven D'Aprano wrote:
>
> Oh great, now you're going to conflict with walrus...
>
> def process(obj:Union[T:=something, List[T]]:=func(x:=expression)+x)->T:
>
> We ought to at least try to avoid clear and obvious conflicts between
> new and existing syntax.
>
> Using `:=` is even worse than `=>`, and `=:` is just *begging* to
> confuse newcomers "why do we have THREE different assignment symbols?"
>
>
You can't have it both ways.  Either you re-use `:=`, or you add a third
symbol.
(There would be no actual conflict with the walrus operator.)
Best wishes
Rob Cliffe

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/63THLQLJW6FRNAY6TFBVD7W3HCKCQDZB/

[Rob Cliffe via Python-ideas]

On 02/12/2021 03:35, Steven D'Aprano wrote:
> On Wed, Dec 01, 2021 at 05:16:34PM +1100, Chris Angelico wrote:
>
>> 1) If this feature existed in Python 3.11 exactly as described, would
>> you use it?
> Yes I would, but probably not as often as counting cases of the "if
> param is None: ..." idiom might lead you to expect.
>
> The problem is, as Neil also pointed out, that it becomes tricky to
> explicitly ask for the default behaviour except by leaving the argument
> out altogether. Not impossible, as Chris mentions elsewhere, you can
> mess about with `*args` or `**kwargs`, but it is decidedly less
> convenient, more verbose, and likely to have a performance hit.
>
> So if I were messing about in the interactive interpreter, I would
> totally use this for the convenience:
>
> def func(L=>[]): ...
>
> but if I were writing a library, I reckon that probably at least half
> the time I'll stick to the old idiom so I can document the parameter:
>
> "If param is missing **or None**, the default if blah..."
>
> I reject Chris' characterisation of this as a hack. There are function
> parameters where None will *never* in any conceivable circumstances
> become a valid argument value, and it is safe to use it as a sentinel.
>
>

There are also circumstances when you start off thinking that None will
never be a valid argument value, but later you have to cater for it. 
Now it does start to look as if you used a hack.
Best wishes
Rob Cliffe

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/KZIOGPBXIFCJQRTUAWCJRHM6TGYC554Z/

[Rob Cliffe via Python-ideas]

On 02/12/2021 07:05, Brendan Barnwell wrote:
> On 2021-12-01 18:35, Chris Angelico wrote:
>> In my reference implementation, there is no object that stores it;
>> it's simply part of the function. A good parallel is the if/else
>> expression:
>>
>> x = float("inf") if z == 0 else y/z
>>
>> Is there an object that represents the if/else expression (or the
>> deferred "y/z" part)? No, although it could be implemented that way if
>> you chose:
>
>     This is not a good parallel.  There is nothing deferred there. 
> The entire line is evaluated when it is encountered and you get a
> result and no part of the if/else expression can ever impact anything
> else again unless, via some external control flow, execution returns
> and re-executes the entire line.  That is not comparable to a function
> default, which is STORED and evaluated later independently of the
> context in which it was originally written (i.e., the function default
> is re-executed but the function definition itself is not re-executed).
>
>     The ternary expression vanishes without a trace by the next line,
> leaving only its evaluated result.  There would be no use in being
> able to access some part of it,

It could be useful for debugging.
Best wishes
Rob Cliffe

> since the whole (i.e., the ternary expression) is completely finished
> by the time you would be able to access it.  This is not the case with
> a function definition.  The function definition leaves behind a
> function object, and that function object needs to "know" about the
> late-bound default as an independent entity so that it can be
> evaluated later.  It is bad for the function to store that late-bound
> default only in some private format for its exclusive future use
> without providing any means for other code to access it as a
> first-class value.
>

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/SCO4G5AWQ47W3DGKSN4WLCN7WR3V6XAA/

[Rob Cliffe via Python-ideas]

On 01/12/2021 15:39, David Mertz, Ph.D. wrote:

However, even if I assume the mythical future PEP never happens, in terms of readability, a WORD is vastly less confusing than a combination of punctuation that has no obvious or natural interpretation like '=>'.  Or rather, I think that spelling is kinda-sorta obvious for the lambda meaning, and the use you want is kinda-sorta similar to a lambda.  So I *do* understand how you get there... but it still seems like much too much line noise for a very minimal need.

Hm.  A word is "vastly less confusing".  OK.  Should Python have been designed with
    x assigned y+1
rather than
    x = y +1
(note that '=' here does not have its "obvious or natural interpretation", viz. "is equal to").

Should we have
    (x becomes y+1)
rather than
    (x := y+1)

Conciseness *is* a virtue, even if it is often trumped by other considerations.
Best wishes
Rob Cliffe

[Rob Cliffe via Python-ideas]

On 01/12/2021 06:16, Chris Angelico wrote:
> I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
> with some additional information about the reference implementation,
> and some clarifications elsewhere.
>
> *PEP 671: Syntax for late-bound function argument defaults*
>
> Questions, for you all:
>

> 1) If this feature existed in Python 3.11 exactly as described, would
> you use it?

Yes, when appropriate.

>
> 2) Independently: Is the syntactic distinction between "=" and "=>" a
> cognitive burden?
>
> (It's absolutely valid to say "yes" and "yes", and feel free to say
> which of those pulls is the stronger one.)

Yes.  Any way PEP 671 is implemented adds to the Python learning curve
and the cognitive burden, by definition.  I don't see how it is
logically possible to answer "No" to this question.
But IMHO it is well worth it.
Providing only one of early-bound and late-bound defaults (and arguably
the less useful one) is a deficiency in Python.  Providing both would be
useful.
New features are added to the language because they are thought to be
useful.  That means that people have to learn about them (even if they
don't write them themselves).  That's life.

>
> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> defaults, (b) referencing things that might have changed, (c)
> referencing other arguments, (d) something else?

I can imagine using it for (a), (b) and (c).  Nothing else springs to
mind at the moment.

>
> 4) If "no" to question 1, is there some other spelling or other small
> change that WOULD mean you would use it? (Some examples in the PEP.)

I answered "yes" to question 1, but I'm not letting that stop me from
reiterating that I think the `=>' arrow is the wrong way round.😁  Hence
my preference for `:=` or `=:`.

>
> 5) Do you know how to compile CPython from source, and would you be
> willing to try this out? Please? :)

It's lazy of me, but like others I doubt that I'll find the time. Sorry.
>
> I'd love to hear, also, from anyone's friends/family who know a bit of
> Python but haven't been involved in this discussion. If late-bound
> defaults "just make sense" to people, that would be highly
> informative.
>
> Any and all comments welcomed. I mean, this is python-ideas after
> all... bikeshedding is what we do best!
>
> The reference implementation currently has some test failures, which
> I'm looking into. I'm probably going to make this my personal default
> Python interpreter for a while, to see how things go.
>
> ChrisA
>
I agree with 3 things that Abe Dillon said in 3 separate posts:

(1) What I really don't like is @param=[] because it puts the emphasis

on the parameter name rather than the act of binding. Not only does it
make it look like @param is a special kind of variable, it also mimics
the *args and **kwargs syntax which makes them seem related.

(2) Yes, we know *why* the hack works. We're all familiar with it. That

doesn't mean it's not a hack.
The bottom line is:
you *don't actually* want the parameter to default to the value of a
sentinel.
you *have* to use that hack because you can't express what you want the
default to actually be.
You're doing something misleading to work around a shortcoming of the
language.
That's a hack.
You have to write something that you don't actually intend.

(3) 2% of functions is a lot of functions.

Best wishes
Rob Cliffe

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/2DKS7HYD5FWDKK4UT3SU3J7BRU7CSNMJ/

[David Mertz, Ph.D.]

On Thu, Dec 2, 2021 at 3:33 AM Chris Angelico <ros...@gmail.com> wrote:

>         But it IS stored!  There is no way for it to be evaluated without it
> being stored!
>

I'm not sure I understand you here. How is the late-bound default
"stored" when one side of a ternary is "not stored"?

This seems like dishonest argument.  I'm not even sure what point you think it is making.

Every time I write a function, everything the function does needs to be STORED.  The body is *stored* in the .__code__ attribute.  Other things are stored in .__annotations__ and elsewhere.  A function is an OBJECT, and everything about it has to be attributes of that object.

>>> def foo(a):

...     b = a + 1

...     print(b)

...

>>> foo.__code__

<code object foo at 0x7f167e539710, file "<ipython-input-7-3c44060a0872>", line 1>

A late binding isn't that one thing about a function that never gets stored, but floats in the ether magically ready to operate on a function call by divine intervention.  It HAS TO describe *something* attached to the function object, doing *something* by some means.

--

Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.

[Steven D'Aprano]

On Thu, Dec 02, 2021 at 11:00:33PM +1100, Chris Angelico wrote:
> On Thu, Dec 2, 2021 at 8:40 PM Steven D'Aprano <st...@pearwood.info> wrote:
> > Depending on the implementation, you *might* be able to inspect the
> > function and see the default expression as some sort of callable
> > function, or evaluatable code object. (That would be nice.)
>
> Unfortunately not, since the default expression could refer to other
> parameters, or closure variables, or anything else from the context of
> the called function. So you won't be able to externally evaluate it.

Why not? Functions can do all those things: refer to other variables, or
closures, or anything else. You can call functions. Are you sure that
this limitation of the default expression is not just a limitation of
your implementation?

> I'm still unsure whether this is a cool feature or an utter abomination:
>
> >>> def f(x=...):
> ... try: print("You passed x as", x)
> ... except UnboundLocalError: print("You didn't pass x")
> ...
> >>> f.__defaults_extra__ = ("n/a",)
> >>> f(42)
> You passed x as 42
> >>> f()
> You didn't pass x

That is absolutely an abomination. If your implementation has the
side-effect that setting a regular early-bound default to Ellipsis makes
the parameter unable to retrieve the default, then the implementation is
fatally broken.

It absolutely is not a feature.


--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/VWZ5SN25QEVVXIYAR43MVRYYTPMANJG3/

[Chris Angelico]

On Fri, Dec 3, 2021 at 12:43 AM David Mertz, Ph.D.
<david...@gmail.com> wrote:
>
> On Thu, Dec 2, 2021 at 3:33 AM Chris Angelico <ros...@gmail.com> wrote:
>>
>> > But it IS stored! There is no way for it to be evaluated without it
>> > being stored!
>> >
>> I'm not sure I understand you here. How is the late-bound default
>> "stored" when one side of a ternary is "not stored"?
>
>
> This seems like dishonest argument. I'm not even sure what point you think it is making.
>

My point was a response to a claim that a late-bound default
fundamentally has to be stored somewhere, yet half of a ternary
conditional isn't. My point was that there is no difference, and they
are both code. Please examine the surrounding context and respond in
more detail, rather than calling my argument "dishonest".

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/GYTVMNI6G4SEYCHILT2AKD23T7CD6AUS/

[Rob Cliffe via Python-ideas]

On 02/12/2021 14:47, Steven D'Aprano wrote:
> On Thu, Dec 02, 2021 at 11:00:33PM +1100, Chris Angelico wrote:
> I'm still unsure whether this is a cool feature or an utter abomination:
>>>>> def f(x=...):
>> ... try: print("You passed x as", x)
>> ... except UnboundLocalError: print("You didn't pass x")
>> ...
>>>>> f.__defaults_extra__ = ("n/a",)
>>>>> f(42)
>> You passed x as 42
>>>>> f()
>> You didn't pass x
> That is absolutely an abomination. If your implementation has the
> side-effect that setting a regular early-bound default to Ellipsis makes
> the parameter unable to retrieve the default, then the implementation is
> fatally broken.
>
> It absolutely is not a feature.
>
>

It's backward incompatible:

15:03:04 R:\>python
Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:20:19) [MSC v.1925 32
bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

>>> def f(x=...):
...     try: print("You passed x as", x)
...     except UnboundLocalError: print("You didn't pass x")
...

>>> f()
You passed x as Ellipsis

So I must agree with Steven that this should not be a feature.
Best wishes
Rob Cliffe

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/DH5QUOPS3KZGDCHEUIM5CSJOCUOAC2KG/

[Chris Angelico]

On Fri, Dec 3, 2021 at 1:53 AM Steven D'Aprano <st...@pearwood.info> wrote:
>
> On Thu, Dec 02, 2021 at 11:00:33PM +1100, Chris Angelico wrote:
> > On Thu, Dec 2, 2021 at 8:40 PM Steven D'Aprano <st...@pearwood.info> wrote:
> > > Depending on the implementation, you *might* be able to inspect the
> > > function and see the default expression as some sort of callable
> > > function, or evaluatable code object. (That would be nice.)
> >
> > Unfortunately not, since the default expression could refer to other
> > parameters, or closure variables, or anything else from the context of
> > the called function. So you won't be able to externally evaluate it.
>
> Why not? Functions can do all those things: refer to other variables, or
> closures, or anything else. You can call functions. Are you sure that
> this limitation of the default expression is not just a limitation of
> your implementation?

def f():
a = 1
def f(b, c=>a+b): return c
a = 2
return f

If there were a function to represent the late-bound default value for
c, what parameters should it accept? How would you externally evaluate
this? And also: what do you gain by it being a function, other than a
lot of unnecessary overhead?

And it is potentially a LOT of unnecessary overhead. Consider this edge case:

def f(a, b=>c:=len(a)): ...

In what context should the name c be bound? If there's a function for
the evaluation of b, then that implies making c a closure cell, just
for the sake of that. Every reference to c anywhere in the function
(not just where it's set to len(a), but anywhere in f()) has to
dereference that.

It's a massive amount of completely unnecessary overhead AND a
difficult question of which parts belong in the closure and which
parts belong as parameters, which means that this is nearly impossible
to define usefully.

> > I'm still unsure whether this is a cool feature or an utter abomination:
> >
> > >>> def f(x=...):
> > ... try: print("You passed x as", x)
> > ... except UnboundLocalError: print("You didn't pass x")
> > ...
> > >>> f.__defaults_extra__ = ("n/a",)
> > >>> f(42)
> > You passed x as 42
> > >>> f()
> > You didn't pass x
>
> That is absolutely an abomination. If your implementation has the
> side-effect that setting a regular early-bound default to Ellipsis makes
> the parameter unable to retrieve the default, then the implementation is
> fatally broken.
>
> It absolutely is not a feature.

That's not what the example shows. It shows that changing dunder
attributes can do this. I'm not sure why you think that the
implementation is as restricted as you imply. The assignment to
__defaults_extra__ is kinda significant here :)

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/CSIIZBRWYOIDTGSVUV67BOIQYMDWNR2U/

[Chris Angelico]

Clearly I shouldn't post code examples without lots and lots of
explanatory comments.

https://www.python.org/dev/peps/pep-0671/#implementation-details

# REDEFINE THE INTERPRETER'S UNDERSTANDING
# OF THE LATE BOUND DEFAULTS FOR THE FUNCTION

f.__defaults_extra__ = ("n/a",)

There, now it's a bit clearer what's going on.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/HFANZOQPA366IT2R3HINCF5VANYT2OTM/

[Steven D'Aprano]

On Wed, Dec 01, 2021 at 09:58:11PM -0600, Abe Dillon wrote:

> My favorite alternative is ?= if people think => and -> are getting
> overly loaded. What I really don't like is @param=[] because it puts the

> emphasis on the parameter name rather than the act of binding. Not only
> does it make it look like @param is a special kind of variable, it also
> mimics the *args and **kwargs syntax which makes them seem related.

But it *is* a special kind of parameter: it is a parameter that uses
late binding for the default value, instead of early binding.

Putting the emphasis on the parameter name is entirely appropriate.

Late bound parameters don't have a different sort of binding to other
parameters. There aren't two kinds of binding:

1. Parameters with no default, and parameters with early bound default,
use the same old type of name binding that other local variables use
(local slots in CPython, maybe a dict for other implementations);

2. and parameters with late bound defaults use a different sort of name
binding, and we need a different syntax to reflect that.

That is wrong: there is only one sort of binding in Python (although it
can have a few different storage mechanisms: slots, cells and dict
namespaces, maybe even others). The storage mechanism is irrelevant.
It's all just name binding, and the implementation details are handled
by the interpreter.

By the time the function body is entered, all the parameters have been
bound to a value. (If there are parameters that don't have a value, the
interpreter raises an exception and you never enter the function body.)
It doesn't matter where those values came from, whether they were passed
in by the caller, or early bound defaults loaded from the cache, or late
bound defaults freshly evaluated. The value is bound to the parameter in
exactly the same way, and you cannot determine where that value came
from.


--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/LYRLA3TH47H4EKOR3NCIUZKXJOICJ5R7/

[Chris Angelico]

On Fri, Dec 3, 2021 at 2:17 AM Steven D'Aprano <st...@pearwood.info> wrote:
>
> On Wed, Dec 01, 2021 at 09:58:11PM -0600, Abe Dillon wrote:
>
> > My favorite alternative is ?= if people think => and -> are getting
> > overly loaded. What I really don't like is @param=[] because it puts the
> > emphasis on the parameter name rather than the act of binding. Not only
> > does it make it look like @param is a special kind of variable, it also
> > mimics the *args and **kwargs syntax which makes them seem related.
>
> But it *is* a special kind of parameter: it is a parameter that uses
> late binding for the default value, instead of early binding.
>
> Putting the emphasis on the parameter name is entirely appropriate.
>
> Late bound parameters don't have a different sort of binding to other
> parameters. There aren't two kinds of binding:
>
> 1. Parameters with no default, and parameters with early bound default,
> use the same old type of name binding that other local variables use
> (local slots in CPython, maybe a dict for other implementations);
>
> 2. and parameters with late bound defaults use a different sort of name
> binding, and we need a different syntax to reflect that.
>
> That is wrong: there is only one sort of binding in Python (although it
> can have a few different storage mechanisms: slots, cells and dict
> namespaces, maybe even others). The storage mechanism is irrelevant.
> It's all just name binding, and the implementation details are handled
> by the interpreter.

In these examples, is the name binding any different?

x = 1
x += 1
(x := 1)
for x in [1]: pass

They use different equals signs, but once the assignment happens, it's
the exact same name binding.

But exactly what IS different about them? Is the name 'x' different?
The value 1? The name binding itself?

> By the time the function body is entered, all the parameters have been
> bound to a value. (If there are parameters that don't have a value, the
> interpreter raises an exception and you never enter the function body.)
> It doesn't matter where those values came from, whether they were passed
> in by the caller, or early bound defaults loaded from the cache, or late
> bound defaults freshly evaluated. The value is bound to the parameter in
> exactly the same way, and you cannot determine where that value came
> from.

Yes, that is correct. And the parameter is the exact same kind of
thing either way, too, so adorning the parameter name is wrong too.
And the expression being evaluated is still just an expression, so
adorning THAT is wrong too. There's nothing about late-bound defaults
that is fundamentally different from the rest of Python, so there's no
obvious "this is the bit to adorn" thing.

In contrast, *args and **kwargs actually change the parameter that
gets received: one gives a sequence, the other a mapping, whereas
leaving off both markers gives you a single value.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/TBEHO47ICQSZIS64ZNDCSGMVFCH4L565/

[Steven D'Aprano]

On Wed, Dec 01, 2021 at 09:00:50PM -0800, abed...@gmail.com wrote:

> Steven D'Aprano

> ""If param is missing **or None**, the default if blah..."

> The bottom line is:
>
> you *don't actually* want the parameter to default to the value of a
> sentinel.

Yes I do. I *do* want to be able to give a convenient sentinel value in
order to explicitly tell the function "give me the default value".

> you *have* to use that hack because you can't express what you want the
> default to actually be.

The point of having default values is so that the caller doesn't have to
express what the default will actually be. If the caller has to express
that value, it's not a default, it's a passed-in argument.

> You're doing something misleading to work around a shortcoming of the
> language.

How is it misleading? The parameter is explicitly documented as taking
None to have a certain effect. None is behaving here as a convenient,
common special constant to trigger a certain behaviour, no different
than passing (for example) buffering=-1 to open() to trigger a very
complex set of behaviour.

(With buffering=-1, the buffer depends on the platform details, and
whether the file is binary, text, and whether or not it is a tty.)

For historical reasons, probably related to C, the default value for
buffering is -1. But it could have just as easily be None, or the string
"default", or an enumeration, or some builtin constant.


--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/QGPPY4JLABDSKICIVJQZ5IRWMXMYAOH4/

[Nicholas Cole]

On Wed, Dec 1, 2021 at 6:18 AM Chris Angelico <ros...@gmail.com> wrote:
>
> I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
> with some additional information about the reference implementation,
> and some clarifications elsewhere.
>
> *PEP 671: Syntax for late-bound function argument defaults*
>
> Questions, for you all:
>
> 1) If this feature existed in Python 3.11 exactly as described, would
> you use it?

I would actively avoid using this feature and discourage people from
using it because:

> 2) Independently: Is the syntactic distinction between "=" and "=>" a
> cognitive burden?

I think that this imposes a significant cognitive burden, not for the
simple cases, but when combined with the more advanced function
definition syntax. I think this has the potential to make debugging
large code-bases much harder.

There is nothing that this proposal makes possible that is not already
possible with more explicit code.

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/3WSRN5NMENAPEUAYQPZCKUW2G43PNJM7/

[Chris Angelico]

On Fri, Dec 3, 2021 at 4:22 AM Nicholas Cole <nichol...@gmail.com> wrote:
> There is nothing that this proposal makes possible that is not already
> possible with more explicit code.

It's worth noting that "explicit" does not mean "verbose". For
instance, this is completely explicit about what it does:

x += 1

It does not conceal what it's doing, yet it uses a very compact
notation to say "augmented addition". The proposal in question uses an
explicit symbol to indicate that the default should be late-bound.

In contrast, a less explicit and much worse proposal might be: "If the
argument default defines a mutable object, construct a new one every
time", so "def f(x=1):" would be early bound and "def f(x=[]):" would
be late-bound. This is implicit behaviour, since it's not stated in
the code which one is which.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/L6PTUVZ33N3EHVCLEGDCDOMB7XGIJRRQ/

[Brendan Barnwell]

On 2021-12-02 01:35, Steven D'Aprano wrote:
>>> > >4) If "no" to question 1, is there some other spelling or other small
>>> > >change that WOULD mean you would use it? (Some examples in the PEP.)
>> >
>> > No. As I mentioned in the earlier thread, I don't support any
>> > proposal in which an argument can "have a default" but that default is not
>> >a first-class Python object of some sort.
> I don't understand this criticism.
>
> Of course the default value will be a first-class Python object of some

> sort.*Every* value in Python is a first-class object. There are no

> machine values or unboxed values, and this proposal will not change
> that.
>

> All that this proposal changes is*when* and*how often* the default

> will be evaluated, not the nature of the value.

As has happened often in these threads, it seems different people mean
different things by "default value".

What you are calling "the default value" is "a thing that is used at
call time if no value is passed for the argument". What I am calling
"the default value" is "a thing that is noted at definition time to be
used later if no value is passed for the argument".

What I'm saying is that I want that "thing" to exist. At the time the
function is defined, I want there to be a Python object which represents
the behavior to be activated at call time if the argument is not passed.
In the current proposal there is no such "thing". The function just
has behavior melded with its body that does stuff, but there is no
addressable "thing" where you can say "if you call the function and the
argument isn't passed were are going to take this
default-object-whatchamacallit and 'use' it (in some defined way) to get
the default value". This is what we already have for early-bound
defaults in the function's `__defaults__` attribute.

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."
--author unknown

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/LZJPLNKD2YOZJMUYFSVFDNOVA6DP6UQL/

[Chris Angelico]

On Fri, Dec 3, 2021 at 6:24 AM Brendan Barnwell <bren...@brenbarn.net> wrote:
> What I'm saying is that I want that "thing" to exist. At the time the
> function is defined, I want there to be a Python object which represents
> the behavior to be activated at call time if the argument is not passed.
> In the current proposal there is no such "thing". The function just
> has behavior melded with its body that does stuff, but there is no
> addressable "thing" where you can say "if you call the function and the
> argument isn't passed were are going to take this
> default-object-whatchamacallit and 'use' it (in some defined way) to get
> the default value". This is what we already have for early-bound
> defaults in the function's `__defaults__` attribute.
>

What would you do with this object? Suppose you have this function:

def f(lst=>[], n=>len(lst)):
...

What can you usefully do with this hypothetical object representing
the default for n? There is no "thing" and no way you could "use" that
thing to predict the value of n without first knowing the full context
of the function.

With early-bound defaults, you can meaningfully inspect them and see
what the value is going to be. With late-bound defaults, should the
default for lst be an empty list? Should the default for n be 0?
Neither is truly correct (but they're wrong in different ways).

Do you actually need that object for anything, or is it simply for the
sake of arbitrary consistency ("we have objects for early-bound
defaults, why can't we have objects for late-bound defaults")?

The value that will be assigned to the parameter does not exist until
the function is actually called, and may depend on all kinds of things
about the function and its context. Until then, there is no value, no
object, that can represent it.

I put to you the same question I put to David Mertz: how is a
late-bound default different from half of a conditional expression?

def f(*args):
lst = args[0] if len(args) > 0 else []
n = args[1] if len(args) > 1 else len(lst)
...

Do the expressions "[]" and "len(lst)" have to have object
representations in this form? If not, why should they need object
representations when written in the more natural way as "lst=>[]"?

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/66MYEHMUSXMAN7AJO2KSMCLUFJ2SOVQC/

[Oscar Benjamin]

On Thu, 2 Dec 2021 at 17:28, Chris Angelico <ros...@gmail.com> wrote:
>
> On Fri, Dec 3, 2021 at 4:22 AM Nicholas Cole <nichol...@gmail.com> wrote:
> > There is nothing that this proposal makes possible that is not already
> > possible with more explicit code.
>
> It's worth noting that "explicit" does not mean "verbose". For
> instance, this is completely explicit about what it does:
>
> x += 1
>
> It does not conceal what it's doing, yet it uses a very compact
> notation to say "augmented addition". The proposal in question uses an
> explicit symbol to indicate that the default should be late-bound.
>
> In contrast, a less explicit and much worse proposal might be: "If the
> argument default defines a mutable object, construct a new one every
> time", so "def f(x=1):" would be early bound and "def f(x=[]):" would
> be late-bound. This is implicit behaviour, since it's not stated in
> the code which one is which.

A bit of an aside but I find it interesting that you pick += for the
example here because there is very much an implicit behaviour with +=
that it mutates in-place or not depending on the mutability of the
object in question (a property that is invisible in the code). For x
+= 1 you can guess that x is a number and remember that all the
standard number types are immutable. Where you have e.g. mutable and
immutable versions of a type though there is no way to know just by
looking at the augmented assignment statement itself:

>>> S1 = {1, 2, 3}
>>> S2 = frozenset(S1)
>>> S3, S4 = S1, S2
>>> S3 |= {4}
>>> S4 |= {4}
>>> S1
{1, 2, 3, 4}
>>> S2
frozenset({1, 2, 3})

Is this implicitness a problem in practice? Usually it isn't but very
occasionally it gives the kind of bug that can send someone banging
their head against a wall for a long time.

--
Oscar

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/4DGKVUH2EYUP4PUUWUVJE5MTIJ3DUFHJ/

[abed...@gmail.com]

Nicholas Cole

"There is nothing that this proposal makes possible that is not already
possible with more explicit code."

There's nothing any of Python's syntax makes possible that is not already possible with Brainfuck or any other language that's Turing complete. The current hacks used to get around the lack of late-bound optional parameters aren't more explicit. They just require more code. If the coder's intent is to have an optional parameter default to an empty list, the most EXPLICIT way to encode that intent would be to have the optional parameter default to an empty list. It's categorically LESS EXPLICIT to bind the parameter to None and add boilerplate code to the body of the function to correct that.

[Eric V. Smith]

I also have this objection to the proposal (among other concerns).

Say I have a function with an early-bound default. I can inspect it and
I can change it. One reason to inspect it is so that I can call the
function with its default values. This is a form of wrapping the
function. I realize "just don't pass that argument when you call the
function" will be the response, but I think in good faith you'd have to
admit this is more difficult than just passing some default value to a
function call.

As far as changing the defaults, consider:

>>> def f(x=3): return x
...
>>> f()
3
>>> f.__defaults__=(42,)
>>> f()
42

The current PEP design does not provide for this functionality for
late-bound defaults.

I realize the response will be that code shouldn't need to do these
things, but I do not think we should be adding features to python that
limit what introspections and runtime modifications user code can do.

A classic example of this is PEP 362 function signature objects. I don't
think we should be adding parameter types that cannot be represented in
a Signature, although of course a Signature might need to be extended to
support new features. Signature objects were added for a reason (see the
PEP), and I don't think we should just say "well, that's not important
for this new feature". Also note that over time we've removed
restrictions on Signatures (see, for example, Argument Clinic). So I
don't think adding restrictions is the direction we want to go in.

Eric

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/A5Y2ANSD52TZMPSS4ODXXHJQKHCFTDPG/

[David Mertz, Ph.D.]

On Thu, Dec 2, 2021 at 2:40 PM Chris Angelico <ros...@gmail.com> wrote:

How is a late-bound default different from half of a conditional expression?

 

 def f(lst=>[], n=>len(lst)): 

def f(*args):
    lst = args[0] if len(args) > 0 else []
    n = args[1] if len(args) > 1 else len(lst)     

Although such is obviously not your intention, I think you have provided a stronger argument against this feature/PEP than any other I've seen raised so far.  Well, maybe just against the specific implementation you have in mind.

You are correct, of course, that the second form does not provide inspectability for `lst` and `n`.  Well, it does, but only in a fairly contorted way of disassembling f.__code__.co_code.  Or maybe with an equally indirect look at the parse tree or something.  The body of a function is very specifically A BODY.

What your proposal/implementation does is put things into the function signature that are simultaneously excluded from direct inspectability as function attributes. Python, like almost all programming languages, makes a pretty clear distinction between function signatures and function bodies.  You propose to remove that useful distinction, or at least weaken it.

For the reasons Eric Smith and others have pointed out, I really WANT to keep inspectability of function signatures.

[abed...@gmail.com]

Steven D'Aprano
"> ""If param is missing **or None**, the default if blah..."

> The bottom line is:
>
> you *don't actually* want the parameter to default to the value of a
> sentinel.

Yes I do. I *do* want to be able to give a convenient sentinel value in
order to explicitly tell the function "give me the default value"."

If you want the parameter to default to an empty list, it's categorically more explicit to have the parameter default to an empty list. It's a hack to say "I wan't this to default to an empty list" then have it actually default to a sentinel value that you'll replace with an empty list later on. That's the opposite of explicit. If you actually wanted to have the parameter default to a sentinel value, why would you then overwrite that value? This isn't a very complicated point.

Steven D'Aprano

"> you *have* to use that hack because you can't express what you want the

> default to actually be.

The point of having default values is so that the caller doesn't have to
express what the default will actually be. If the caller has to express
that value, it's not a default, it's a passed-in argument."

None of my comment had anything to do with the caller's perspective. I know how optional parameters work. I'm specifically talking about the programmer's ability to map their intent to code and the ability of the would-be reader of the code (NOT CALLER) to decipher that intent.

If my intent is that I want a parameter to default to an empty list, I wouldn't assign it to an object called "monkey_pants" by default then add code to the body of the function to check if it's "monkey_pants".

"But wait", I hear you say, "assigning my optional parameter to 'monkey_pants' isn't a hack because I know will *never* in any conceivable circumstances become a valid argument value!"
"assigning a value to 'monkey_pants' is how I explicitly communicate tell the function "give me the default value"!"

Those are both quotes from you with 'None' replaced with 'monkey_pants'.

Steven D'Aprano
"How is it misleading?"

Your way of having a parameter default to an empty list (or whatever) is by having the parameter default to None. That's how it's misleading. I know how the hack works. I know why the hack works. I know how default parameters work. You can spare trying to explain all that again. What you can't explain is why it's more explicit to have a parameter default to a sentinel value when the actual intent is to have it default to a mutable value. If it were more explicit, then why not replace `def func(x=0)` with `def func(x=SENTINEL): if x is SENTINEL: x = 0`?

I'm at a loss for ways to explain my position any clear than that.

[abed...@gmail.com]

Steven D'Aprano

"> My favorite alternative is ?= if people think => and -> are getting

> overly loaded. What I really don't like is @param=[] because it puts the

> emphasis on the parameter name rather than the act of binding. Not only
> does it make it look like @param is a special kind of variable, it also
> mimics the *args and **kwargs syntax which makes them seem related.

But it *is* a special kind of parameter: it is a parameter that uses
late binding for the default value, instead of early binding.

Putting the emphasis on the parameter name is entirely appropriate.

Late bound parameters don't have a different sort of binding to other

parameters. There aren't two kinds of binding:"

Again: the important part is the *binding*, not the name of the parameter.
Right now, the order of reading something like `def load_tweets(... hash_tags: List[str] = None, ...)` goes like this:

1) there's a parameter named 'hash_tags'
2) it's a list of strings
3) it has a default value (bound at function declaration)

4) the default value is None

With `def load_tweets(... @hash_tags: List[str] = [], ...)` it goes like:

1) There's some parameter bound late
2) It's a parameter named 'hash_tags'
3) It's a list of strings
4) It has a default value (of course because of information we learned 3 steps ago)
5) The default value is an empty list

With `def load_tweets(... hash_tags: List[str] => [], ...)` it goes like:

1) there's a parameter named 'hash_tags'
2) it's a list of strings
3) it has a default value (bound at call-time)

4) the default value is None

Notice how it makes way more sense to put related parts closer together rather than saying "here's a little information. Now put a pin in that: we'll get back to it later on".

To drive the point home: The @ and = signs are connected, but not visually, which is confusing. You couldn't write @param without an '=' somewhere down the line:
`def load_tweets(... @hash_tags, ...)  # Error!`

I probably should have been more clear about what I meant when I said "it make[s] it look like @param is a special kind of variable, it also mimics the *args and **kwargs syntax which makes them seem related." Of course the way the parameter is bound *is* special making it, in the ways you pointed out, a "special kind of variable", however, again: the specialness is all about the binding of the variable, and: unlike *args and **kwargs, @param doesn't imply any specific type (in the 'type(kwargs)' sense) for the variable. We know that args and kwargs are an iterable and mapping, respectively.

The other question is: how important is the information that a parameter is late-binding. I've always been against the coding standard of making enums and constants ALL_CAPS because it makes code shouty and obnoxious just as it makes any other communication shouty and obnoxious. It doesn't seem like the fact that something is an enum or constant is important enough to warrant anything more than some syntax coloring from my IDE (if that). People always point out that the fact that something is an enum or constant is, in fact, information, and how else would they possibly communicate that information? The answer is: you communicate it by declaring an enum or constant and stop shouting at me! There's other information I you could put in your variable names that most sane people don't. You don't always append the name of the class of an object to the variable's name. You say 'hash_tags' not 'hash_tags_list_of_strings'.

In the same vein, I also don't see the need here to make a big fuss about the binding behavior of a parameter such that it *must* be up-front ahead of the name, annotation, and other binding info. It may seem like a big deal now because it's new and scary so we must prefix it with "HERE_BE_DRAGONS!!!", but years down the line when late-binding params are common place, it'll be a wart.

[Brendan Barnwell]

On 2021-12-02 00:31, Chris Angelico wrote:
> Here's how a ternary if looks:
>
>>>> >>>def f(n):
> ... return 0 if n == 0 else 42/n
> ...
>>>> >>>dis.dis(f)
> 2 0 LOAD_FAST 0 (n)
> 2 LOAD_CONST 1 (0)
> 4 COMPARE_OP 2 (==)
> 6 POP_JUMP_IF_FALSE 6 (to 12)
> 8 LOAD_CONST 1 (0)
> 10 RETURN_VALUE
> >> 12 LOAD_CONST 2 (42)
> 14 LOAD_FAST 0 (n)
> 16 BINARY_TRUE_DIVIDE
> 18 RETURN_VALUE
>
> The "42/n" part is stored in f.__code__.co_code as the part that says
> "LOAD_CONST 42, LOAD_FAST n, BINARY_TRUE_DIVIDE". It's not an object.
> It's just code - three instructions.
>
> Here's how (in the reference implementation - everything is subject to
> change) a late-bound default looks:
>
>>>> >>>def f(x=>[]): print(x)
> ...
>>>> >>>dis.dis(f)
> 1 0 QUERY_FAST 0 (x)
> 2 POP_JUMP_IF_TRUE 4 (to 8)
> 4 BUILD_LIST 0
> 6 STORE_FAST 0 (x)
> >> 8 LOAD_GLOBAL 0 (print)
> 10 LOAD_FAST 0 (x)
> 12 CALL_FUNCTION 1
> 14 POP_TOP
> 16 LOAD_CONST 0 (None)
> 18 RETURN_VALUE
>
> The "=>[]" part is stored in f.__code__.co_code as the part that says
> "QUERY_FAST x, and if false, BUILD_LIST, STORE_FAST x". It's not an
> object. It's four instructions in the bytecode.
>
> In both cases, no part of the expression is ever re-executed. I'm not
> understanding the distinction here. Can you explain further please?

Your explanation exactly shows how it IS re-executed. I'm not totally
clear on this disassembly since this is new behavior, but if I
understand right, BUILD_LIST is re-executing the expression `[]` and
STORE_FAST is re-assigning it to x. The expression `[]` is
syntactically present in the function definition but its execution has
been shoved into the function body where it may be re-executed many
times (any time the function is called without passing a value).

What do you mean when you say it is not re-executed? Is it not the
case that `[]` is syntactically present in the `def` line (which is
executed only once, and whose bytecode is not shown) yet its
implementation (BUILD_LIST) is in the function body, which may be
executed many times? How is the BUILD_LIST opcode there not being
re-executed on later calls of the function?

Perhaps what you are saying is that what is stored is not the literal
string "[]" but bytecode that implements it? That distinction is
meaningless to me. The point is that you wrote `[]` once, in a line
that is executed once (the function definition itself), but the `[]` is
executed many times, separately from the function definition.

Another way to put it is, again, the examples are not parallel. In
your first example the ternary expression is (syntactically) part of the
function BODY, so of course it appears in the disassembly. In your
second example, the late-bound default is not in the body, it is in the
signature. The disassembly is only showing the bytecode of the function
body, but the late-bound default is syntactically enmeshed with the
function DEFINITION. So I don't want the late-bound default code to be
re-executed unless the function is re-defined. Or, if you're going to
store that `x = []`, I don't want it to be "stored" by just shoving it
in the function body, I want it to be some kind of separate object.

Here's an example that may make my point clearer.

some_function(x+1, lambda x: x+2)

This is our best current approximation to some kind of "late
evaluation". The first argument, `x+1`, is evaluated before the
function is called, and the function gets only the result. The second
argument is also of course evaluated before the function is called, but
what is evaluated is a lambda; the `x+2` is not evaluated. But the idea
of "evaluate x+2 later" is encapsulated in a function object. Without
knowing anything about `some_function`, I still know that there is no
way it can ever evaluate `x+2` without going through the interface of
that function object. There is no way to pass, as the second argument
to `some_function` some kind of amorphous "directive" that says
"evaluate x+2 later" without wrapping that directive up into some kind
of Python object.

Perhaps the fundamental point that I feel you're missing about my
position is that a ternary expression does not have a "definition" and a
"body" that are executed at separate times. There is just the whole
ternary expression and it is evaluated all at once. Thus there can be
no parallel with functions, which do have a separation between
definition time and call time. Indeed, it's because these are separate
that late vs. early binding of defaults is even a meaningful concept for
functions (but not for ternary expressions).

So what I am saying is if I see a line like this:

def f(a=x+1, b@=x+2):

The x+2 is syntactically embedded within the `def` line, that is, the
function definition (not the body). Thus there are only two kinds of
semantics that are going to make me happy:

1) That `x+2`(or any bytecode derived from it, etc.) will never be
re-executed unless the program execution again reaches the line with the
`def f` (which is what we have with early-bound defaults)
2) A Python object is created that encapsulates the expression `x+2`
somehow and defines what it means to "evaluate it in a context" (e.g.,
by referencing local variables in the scope where it is evaluated)

Maybe another way to say this is just "I want any kind of late-bound
default to really be an early-bound default whose value is some object
that provides a way to evaluate it later". (I'm trying to think of
different ways to say this because it seems what I'm saying is not clear
to you. :-)

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."
--author unknown

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/IQ7C4MXF25KJ7HXAMTPLOBFQK2N42WU2/

[abed...@gmail.com]

"The caller cannot explicitly ask for default behaviour except by omitting the parameter.  This can be very annoying to set up when the parameter values are provided from other places, e.g.,

if need_x:

  # do lots of stuff

  x = whatever

else:

  # do more

  x = None

f(x=x, ...)  # easy, but with this PEP, you would have to find a way to remove x from the parameter list.  Typically, removing a parameter from a dynamically-created parameter list is hard."

If removing a parameter from a dynamically-created parameter list is hard, maybe that's a tools problem. Maybe we should consider crafting better tools for such tasks. It also sounds like a lot of people believe the inspectability of callables that use late-bound defaults is a point of contention. Maybe there should be more inspection hooks than described in the implementation details section of the PEP?

On Wednesday, December 1, 2021 at 3:56:35 AM UTC-6 miste...@gmail.com wrote:

On Wednesday, December 1, 2021 at 1:18:33 AM UTC-5 Chris Angelico wrote:

I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
with some additional information about the reference implementation,
and some clarifications elsewhere.

*PEP 671: Syntax for late-bound function argument defaults*

Questions, for you all:

1) If this feature existed in Python 3.11 exactly as described, would
you use it?

No, I would avoid for the sake of readers.  Anyway, it can't be used in any public projects until thy drop support for 3.10, which is many years off.

Also, I think this question is quite the biased sample on python-ideas. Please consider asking this to less advanced python users, e.g., reddit.com/r/python or learnpython. 

2) Independently: Is the syntactic distinction between "=" and "=>" a
cognitive burden?

(It's absolutely valid to say "yes" and "yes", and feel free to say
which of those pulls is the stronger one.)

 Yes.

3) If "yes" to question 1, would you use it for any/all of (a) mutable
defaults, (b) referencing things that might have changed, (c)
referencing other arguments, (d) something else?

4) If "no" to question 1, is there some other spelling or other small
change that WOULD mean you would use it? (Some examples in the PEP.)

No. 

 

5) Do you know how to compile CPython from source, and would you be

willing to try this out? Please? :)

I'd love to hear, also, from anyone's friends/family who know a bit of
Python but haven't been involved in this discussion. If late-bound
defaults "just make sense" to people, that would be highly
informative.

Any and all comments welcomed. I mean, this is python-ideas after
all... bikeshedding is what we do best! 

 

This PEP has a lot of interesting ideas.  I still think that none-aware operators (PEP 505) are an easier, more readable general solution to binding calculated default values to arguments succinctly.  I think the problems with this idea include:

* The caller cannot explicitly ask for default behaviour except by omitting the parameter.  This can be very annoying to set up when the parameter values are provided from other places, e.g.,

if need_x:

  # do lots of stuff

  x = whatever

else:

  # do more

  x = None

f(x=x, ...)  # easy, but with this PEP, you would have to find a way to remove x from the parameter list.  Typically, removing a parameter from a dynamically-created parameter list is hard.

* The function code becomes unreadable if the parameter-setting code is long.  Having a long piece of code inside the function after "if parameter is None" is just fine.  Having none-aware operators would make such code more succinct.

* People nearly always avoid writing code in the parameter defaults themselves, and this new practice adds a lot of cognitive load.  E.g., people rarely write:

def f(x: int = 1+g()) -> None: ...

Parameter lists are already busy enough with parameter names, annotations, and defaults.  We don't need to encourage this practice.

In short, I think this is a creative idea, a great exploration.  While optional parameters are common, and some of them have defaults that are calculated inside the function, my feeling is that people will continue to set their values inside the function.

Best,

Neil

[Chris Angelico]

It's only as implicit as every other operator. For instance, when you
write "x + y", that might call type(x).__add__(y), and it might call
type(y).__radd__(x). Is that implicit behaviour? It's very clearly
defined (as is when each will happen).

If there is no __iadd__ method, then += will fall back on + and =.

I think you're confusing "implicit behaviour" with "polymorphic
behaviour", which is a strong tenet of pretty much every modern
object-oriented programming language. The precise behaviour depends on
the types of the objects involved. That's not a problem; it's a
spectacularly useful feature!

And yes, sometimes complexity leads to banging heads on walls. If
there's some weird bug in an __iadd__ method, it can be annoyingly
difficult to track down. But ultimately, it's not that difficult to
figure out exactly what a line of code does. (It's just impractical to
do that simultaneously for every line of code.)

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/WTO36OLWOI7MFBCSCQPUML6MTTEMJLXU/

[Chris Angelico]

1) I want to call this function
2) I may want to not pass this argument
3) Ah, perfect! I will pass this argument with a value of somemod._SENTINEL.

Or alternatively:

1) I want to call this function.
2) Prepare a dictionary of arguments. Leave out what I don't want.
3) If I want to pass this argument, add it to the dictionary.

This way doesn't require reaching into the function's private
information to use a sentinel. Yes, it may be a tad more difficult
(though not VERY much), but you're also avoiding binding yourself to
what might be an implementation detail.

> As far as changing the defaults, consider:
>
> >>> def f(x=3): return x
> ...
> >>> f()
> 3
> >>> f.__defaults__=(42,)
> >>> f()
> 42
>
> The current PEP design does not provide for this functionality for
> late-bound defaults.

Remember, though: the true comparison should be something like this:

_SENTINEL = object()
def f(x=_SENTINEL):
if x is _SENTINEL: x = []
return x

Can you change that from a new empty list to something else? No. All
you can do, by mutating the function's dunders, is change the
sentinel, which is actually irrelevant to the function's true
behaviour. You cannot change the true default.

Consider also this form:

default_timeout = 500
def connect(s, timeout=default_timeout): ...
def read(s, timeout=default_timeout): ...
def write(s, msg, timeout=default_timeout): ...

You can now, if you go to some effort, replace the default in every
function. Or you can do this, and not go to any effort at all:

def read(s, timeout=>default_timeout): ...

The true default is now exactly what the function signature says. And
if you really want to, you CAN change read.__defaults__ to have an
actual early-bound default, which means it will then never check the
default timeout.

Introspection is no worse in this way than writing out the code
longhand. It is significantly better, because even though you can't
change it from a latebound default_timeout to a latebound
read_timeout, you can at least see the value with external tools. You
can't see that if the default is replaced in the body of the function.

> I realize the response will be that code shouldn't need to do these
> things, but I do not think we should be adding features to python that
> limit what introspections and runtime modifications user code can do.

The response is more that the code CAN'T do these things, by
definition. To the extent that you already can, you still can. To the
extent that you should be able to, you are still able to. (And more.
There are things you're capable of with PEP 671 that you definitely
shouldn't do in normal code.)

> A classic example of this is PEP 362 function signature objects. I don't
> think we should be adding parameter types that cannot be represented in
> a Signature, although of course a Signature might need to be extended to
> support new features. Signature objects were added for a reason (see the
> PEP), and I don't think we should just say "well, that's not important
> for this new feature". Also note that over time we've removed
> restrictions on Signatures (see, for example, Argument Clinic). So I
> don't think adding restrictions is the direction we want to go in.

Same again. If you consider the equivalent to be a line of code in the
function body, then the signature has become MASSIVELY more useful.
Instead of simply seeing "x=<object object at 0x7fba1b318690>", you
can see "x=>[]" and be able to see what the value would be. It's
primarily human-readable (although you could eval it), but that's
still a lot better than seeing a meaningless sentinel. And yes, you
can make help() more readable by using a nicely named sentinel, but
then you have to go to a lot more effort in your code, worry about
pickleability, etc, etc. Using a late-bound default lets you see the
true default, not a sentinel.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/6BT6WYMU5RB63UQX5WWDLABRNARWO4BU/

[Chris Angelico]

On Fri, Dec 3, 2021 at 8:07 AM David Mertz, Ph.D. <david...@gmail.com> wrote:
>
> On Thu, Dec 2, 2021 at 2:40 PM Chris Angelico <ros...@gmail.com> wrote:
>>
>> How is a late-bound default different from half of a conditional expression?
>
>
>>
>> def f(lst=>[], n=>len(lst)):
>>
>>
>> def f(*args):
>> lst = args[0] if len(args) > 0 else []
>> n = args[1] if len(args) > 1 else len(lst)
>
>
> Although such is obviously not your intention, I think you have provided a stronger argument against this feature/PEP than any other I've seen raised so far. Well, maybe just against the specific implementation you have in mind.
>

Fascinating.

> You are correct, of course, that the second form does not provide inspectability for `lst` and `n`. Well, it does, but only in a fairly contorted way of disassembling f.__code__.co_code. Or maybe with an equally indirect look at the parse tree or something. The body of a function is very specifically A BODY.
>
> What your proposal/implementation does is put things into the function signature that are simultaneously excluded from direct inspectability as function attributes. Python, like almost all programming languages, makes a pretty clear distinction between function signatures and function bodies. You propose to remove that useful distinction, or at least weaken it.
>

Actually, no. I want to put the default arguments into the signature,
and the body in the body. The distinction currently has a technical
restriction that means that, in certain circumstances, what belongs in
the signature has to be hacked into the body. I'm trying to make it so
that those can be put where they belong.

> For the reasons Eric Smith and others have pointed out, I really WANT to keep inspectability of function signatures.
>

Here's what you get:

>>> def f(lst=>[], n=>len(lst)): ...
...
>>> f.__defaults_extra__
('[]', 'len(lst)')

String representation, but exactly what the default is. Any tool that
inspects a signature (notably, help() etc) will be able to access
this.

Now write the function the other way, and show me how easy it is to
determine the behaviour when arguments are omitted. What can you
access?

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/2RZBMRT5RPHCSRW67GPGEF44E2SHG4ZB/

[Abe Dillon]

If the coder's intent is to have an optional parameter default to an empty list, the most EXPLICIT way to encode that intent would be to have the optional parameter default to an empty list. It's categorically LESS EXPLICIT to bind the parameter to None and add boilerplate code to the body of the function to correct that.

I think it's a nearly philosophical distinction but right now there is no such thing as “an optional parameter defaulting to an empty list”. Or rather there is, but it's not what you really want “a different empty list for every call of the function”.

It's not philosophical. It's pretty cut and dry: If you want a parameter to default to an empty list, it's objectively more explicit to make the parameter default to an empty list than to make it default to a sentinel value and assign it to an empty list later. Of course that capability doesn't exist yet, that's what this whole discussion is about.

To me, it's not absurd that “creating a new object and binding the parameter to it” is sufficiently complex that we don't cram it into the function signature. It's fine to just have code in the function body... 

It may not be absurd to you, but to me, the function signature is the correct place to put all things related to the function signature, the function body is for business logic. You're argument about "cramming things in the function signature" is more about practical constraints like keeping things on one line which is less important to me than putting things where they belong. You can always expand the signature, which is a practice I wish more people did.

Instead of this:

def my_func(param: Annotation[stuff], other_param: MoreAnnotation[things], another_param: SomeType = default_value, yet_another_param: EvenMoreAnnotation = another_default) -> ReturnType: 
     # code goes here ...

just do this:

def my_func(
            param: Annotation[stuff],
            other_param: MoreAnnotation[things],
            another_param: SomeType = default_value,
            yet_another_param: EvenMoreAnnotation = another_default) -> ReturnType:
    # code goes here ...

The repulsion against "cram[ing] [defaults] into the function signature" is more a problem with long function signatures than anything else. You can always use the usual suspects to make it more readable:

instead of this:

def my_func(param: SomeType => long and complicated logic that's hard to read and fits too much in one line) -> ReturnValue: 
    ... 

just do this: 

def _provide_default() -> SomeType: 
    the same logic as above 

    but more readable

    and better organized

def my_func(param: SomeType => _provide_default()) -> ReturnValue:
    ...

especially since there does not seem to be an overwhelming consensus for either the concept or the proposed syntaxes.

The above explanation of why it's objectively more explicit is not subject to the consensus of the python ideas list. It's a refutation of the argument that current familiar work-arounds are somehow more explicit. That somehow binding to a sentinel is actually an explicit way of "telling the function you want it to provide a default value" rather than a work-around that's so familiar that you no longer even question if it's all that explicit in the first place.

(Also maybe not calling what we have been happily (at least for some of us) doing for years "a hack" would be a good way to have a more civil discussion)

Why? It's not a personal attack. The whole point is to prod people into rethinking what they've become inured to.

On Thu, Dec 2, 2021 at 3:11 PM Evpok Padding <evpok....@gmail.com> wrote:

On Thu, 2 Dec 2021, 21:48 abed...@gmail.com, <abed...@gmail.com> wrote:

Nicholas Cole

"There is nothing that this proposal makes possible that is not already
possible with more explicit code."

There's nothing any of Python's syntax makes possible that is not already possible with Brainfuck or any other language that's Turing complete. The current hacks used to get around the lack of late-bound optional parameters aren't more explicit. They just require more code. If the coder's intent is to have an optional parameter default to an empty list, the most EXPLICIT way to encode that intent would be to have the optional parameter default to an empty list. It's categorically LESS EXPLICIT to bind the parameter to None and add boilerplate code to the body of the function to correct that.

I think it's a nearly philosophical distinction but right now there is no such thing as “an optional parameter defaulting to an empty list”. Or rather there is, but it's not what you really want “a different empty list for every call of the function”. To me, it's not absurd that “creating a new object and binding the parameter to it” is sufficiently complex that we don't cram it into the function signature. It's fine to just have code in the function body, especially since there does not seem to be an overwhelming consensus for either the concept or the proposed syntaxes.

(Also maybe not calling what we have been happily (at least for some of us) doing for years "a hack" would be a good way to have a more civil discussion)

Best

E

[Chris Angelico]

Ah, I think I get you. The problem is that code is in the def line but
is only executed when the function is called, is that correct? Because
the code would be "re-executed" just as much if it were written in the
function body. It's executed (at most) once for each call to the
function, just like the ternary's side is.

I suppose that's a consideration, but it's not nearly as strong in
practice as you might think. A lot of people aren't even aware of the
difference between compilation time and definition time (even people
on this list have made that mistake). Function default args are
executed when the function is defined, not when it's called, and
that's something that changes with this proposal; but there are many
other subtleties to execution order and timing that don't really
matter in practice.

Perhaps the key point here is to consider function decorators. We
could avoid them altogether:

def f():
@deco
def g(x): ...

def h(x): ...
h = deco(h)

But as well as having the name replication problem, this buries
important information down in the body of the surrounding code, rather
than putting it at the signature of g/h where it belongs. Even though,
semantically, this is actually part of the body of f, we want to be
able to read it as part of the signature of g. Logically and
conceptually, it is part of the signature. Now compare these two:

def f2():
_SENTINEL = object()
def g(x=_SENTINEL):

if x is _SENTINEL: x = []

...

def h(x=>[]):
...

Which one has its signature where its signature belongs? Yes,
semantically, the construction of the empty list happens at function
call time, not at definition time. But what you're saying is: if there
are no args passed, behave as if a new empty list was passed. That's
part of the signature.

In neither case will you find an object representing the expression []
in the function's signature, because that's not an object, it's an
instruction to build an empty list. In the case of g, you can find a
meaningless and useless object stashed away in __defaults__, but that
doesn't tell you anything about the true behaviour of the function. At
least in the case of h, you can find the descriptive string "[]"
stashed there, which can tell a human what's happening.

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/LZLKSGJ7XSVRUBWHDXU2SA4YRZ5SUZBX/

[abed...@gmail.com]

There was a typo in my post and I want to avoid getting into a pedantic tangent about how to properly indent multi-line function signatures, so hopefully this will "nip it in the bud", so to speak. 

When I wrote:

def my_func(
            param: Annotation[stuff],
            other_param: MoreAnnotation[things],
            another_param: SomeType = default_value,
            yet_another_param: EvenMoreAnnotation = another_default) -> ReturnType:
    # code goes here ...

I meant the parameter list to be at two indentation levels so it's visually distinguishable from the function body at one indentation level. I accidentally put the parameter list at three indentation levels which happen to align with the open parentheses after `def my_func` which is not what I wanted to suggest as good practice because then functions with long names would have their parameter list indented way far over and it would hurt readability. The following is closer to what I intended: 

def my_func(
        param: Annotation[stuff],
        other_param: MoreAnnotation[things],
        another_param: SomeType = default_value,
        yet_another_param: EvenMoreAnnotation = another_default) -> ReturnType:
    # code goes here ...

[Abe Dillon]

Lol! Now the formatting is just trolling me!

Note to self: Gmail > Google Groups > Cell Phone

_______________________________________________

Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/O5GDCBMNUEPTSYS2AVZOFY4IUZAWMXFH/

[Steven D'Aprano]

On Fri, Dec 03, 2021 at 02:10:12AM +1100, Chris Angelico wrote:

> > > Unfortunately not, since the default expression could refer to other
> > > parameters, or closure variables, or anything else from the context of
> > > the called function. So you won't be able to externally evaluate it.
> >
> > Why not? Functions can do all those things: refer to other variables, or
> > closures, or anything else. You can call functions. Are you sure that
> > this limitation of the default expression is not just a limitation of
> > your implementation?
>
> def f():
> a = 1
> def f(b, c=>a+b): return c
> a = 2
> return f
>
> If there were a function to represent the late-bound default value for
> c, what parameters should it accept?

I'm not saying that it *must* be a function. It could be a bare code
object, that is `eval()`ed. Or something completely new. Dunno.

But you're saying something is impossible, and that seems implausible to
me, because things that seems *very similar* are totally possible.

> How would you externally evaluate this?

inner = f() # Get the inner function.
default = inner.__code__.__late_defaults__.wibble[1] # whatever
try:
value = default()
except NameError:
# Well, what did you expect to happen?
value = eval(default.__code__, globals(), {'a': 101, 'b': 202})


Or something. The point is, rather than dismissing the possibility
outright, this should be something we discuss, and carefully consider,
before the PEP is complete.

> And also: what do you gain by it being a function, other than a
> lot of unnecessary overhead?

Nicer introspection.

Brendan goes from strongly opposed to the PEP to its biggest and most
tireless supporter *wink*

Cleaner separation of concerns: the defaults get handled independently
of the function body.

Plays nice with other tools that (say) use byte-code manipulation on
the function body.

> And it is potentially a LOT of unnecessary overhead. Consider this edge case:
>
> def f(a, b=>c:=len(a)): ...
>
> In what context should the name c be bound?

The obvious (which is not necessarily correct) answer is, the same scope
that the expression is evaluated in, unless its declared global or
nonlocal. (Which is only relevant if the function f() is nested in
another function.)

With regular defaults, the expression is evaluated at function
definition time, and c gets bound to the surrounding scope.

With late-bound defaults, the expression is evaluated at function call
time, in the scope of f()'s locals. So c would be a local.

Or the other obvious answer is that c will always be in the surrounding
scope, for both early and late bound defaults.

Consider the existence of walrus expressions in comprehensions:


>>> def demo(a, b=((w:=i**2)*str(w) for i in range(5))):
... return b
...
>>> it = demo(None)
>>> next(it)
''
>>> next(it)
'1'
>>> next(it)
'4444'
>>> w
4

So there is precedent for having function-like entities (in this case, a
comprehension) exposing their walrus variables in the surrounding scope.

The third obvious answer is that if either the decision or the
implementation is really too hard, then make it a syntax error for now,
and revisit it in the future.

> If there's a function for
> the evaluation of b, then that implies making c a closure cell, just
> for the sake of that. Every reference to c anywhere in the function
> (not just where it's set to len(a), but anywhere in f()) has to
> dereference that.

Okay. Is this a problem?

If it really is a problem, then make it a syntax error to use walrus
expressions inside late bound defaults.

> It's a massive amount of completely unnecessary overhead AND a
> difficult question of which parts belong in the closure and which
> parts belong as parameters, which means that this is nearly impossible
> to define usefully.

I've given you two useful definitions.

Its not clear what overhead you are worried about.

Accessing variables in cells is almost as fast as accessing locals, but
even if they were as slow as globals, premature optimization is the root
of all evil. Globals are fast enough.

Or are you worried about the memory overhead of the closures? The extra
cost of fetching and calling the functions when evaluating the defaults?
None of these things seem to be good reasons to dismiss the idea that
default expressions should be independent of the function body.

"Using a walrus expression in the default expression will make your
function 3% slower and 1% larger, so therefore we must not make the
default expression an introspectable code object..."

> > > I'm still unsure whether this is a cool feature or an utter abomination:
> > >
> > > >>> def f(x=...):
> > > ... try: print("You passed x as", x)
> > > ... except UnboundLocalError: print("You didn't pass x")
> > > ...
> > > >>> f.__defaults_extra__ = ("n/a",)
> > > >>> f(42)
> > > You passed x as 42
> > > >>> f()
> > > You didn't pass x

[...]

> That's not what the example shows. It shows that changing dunder
> attributes can do this. I'm not sure why you think that the
> implementation is as restricted as you imply. The assignment to
> __defaults_extra__ is kinda significant here :)

Ah, well that is not so clear to people who aren't as immersed in the
implementation as you :-)

Messing about with function dunders can do weird shit:

>>> def func(a=1, b=2):
... return a+b
...
>>> func.__defaults__ = (1, 2, 3, 4, 5)
>>> func()
9

I wouldn't worry about it.


--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/2K62MH3JAJANDUTTTH46RK76ESV4MREM/

[Chris Angelico]

How, with external calling, are you going to know which name
references to look up, and where to get their values from? This is
already a virtually impossible task in Python, which is why f-strings
cannot be implemented as str.format(**something) for any known meaning
of "something". You cannot get your *current* variable set, much less
the available variables in some other context.

> > And it is potentially a LOT of unnecessary overhead. Consider this edge case:
> >
> > def f(a, b=>c:=len(a)): ...
> >
> > In what context should the name c be bound?
>

> With late-bound defaults, the expression is evaluated at function call
> time, in the scope of f()'s locals. So c would be a local.

In other words, if you're trying to evaluate b's default externally,
you have to set c in a context that doesn't even exist yet. Is that
correct?

> The third obvious answer is that if either the decision or the
> implementation is really too hard, then make it a syntax error for now,
> and revisit it in the future.

Everything's perfectly well defined, save that you can't externally
evaluate the defaults. You can quite happily use the walrus in a
late-bound default, and it'll bind to the function's locals. (Though I
don't recommend it. I think that that's going to make for
less-readable code than alternatives. Still, it's perfectly legal and
well-defined.)

> > It's a massive amount of completely unnecessary overhead AND a
> > difficult question of which parts belong in the closure and which
> > parts belong as parameters, which means that this is nearly impossible
> > to define usefully.
>
> I've given you two useful definitions.
>
> Its not clear what overhead you are worried about.
>
> Accessing variables in cells is almost as fast as accessing locals, but
> even if they were as slow as globals, premature optimization is the root
> of all evil. Globals are fast enough.
>
> Or are you worried about the memory overhead of the closures? The extra
> cost of fetching and calling the functions when evaluating the defaults?
> None of these things seem to be good reasons to dismiss the idea that
> default expressions should be independent of the function body.

The problem is that you're trying to reference cell variables in a
closure that might not even exist yet, so *just in case* you might
have nonlocals, you have to construct a closure... or find an existing
one. Ill-defined and inefficient.

> > That's not what the example shows. It shows that changing dunder
> > attributes can do this. I'm not sure why you think that the
> > implementation is as restricted as you imply. The assignment to
> > __defaults_extra__ is kinda significant here :)
>
> Ah, well that is not so clear to people who aren't as immersed in the
> implementation as you :-)

Maybe, but I did think that the fact that I was assigning to a dunder
should be obvious to people who are reading here :)

> Messing about with function dunders can do weird shit:
>
> >>> def func(a=1, b=2):
> ... return a+b
> ...
> >>> func.__defaults__ = (1, 2, 3, 4, 5)
> >>> func()
> 9
>
> I wouldn't worry about it.
>

Good, so, not an abomination. (In your opinion. Others are, of course,
free to abominate as they please.)

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/ZO7QY33762WUCNGXULVBH6ZCSWOTHKXK/

[Steven D'Aprano]

On Thu, Dec 02, 2021 at 01:20:27PM -0800, abed...@gmail.com wrote:

> If you want the parameter to default to an empty list, it's categorically
> more explicit to have the parameter default to an empty list.

You've gone from talking about *my* intentions, which you got wrong, to
an argument about which is "more explicit".

In the simple example of the default being a new empty list:

# Using Chris' preferred syntax, not mine
def func(arg=>[]):

then I agree: once you know what the syntax means, then it is more
explicit at telling the reader that the default value is an empty list.
Great! This sort of thing is the primary use-case of the feature.

But let's talk about more complex examples:

def open(filename, mode='r', buffering=-1): ...

How would you make that "more explicit"?

def open(filename, mode='r', buffering=>(
1 if 'b' not in mode
and os.fdopen(os.open(filename)).isatty()
else _guess_system_blocksize() or io.DEFAULT_BUFFER_SIZE
)
): ...

Too much information! We're now drowning in explicitness.

Now how do I, the caller, *explicitly* indicate that I want to use that
system-dependent value? I just want a nice, simple value I can
explicitly pass as buffering to say "just use the default".

with open('myfile.txt', buffering= what? ) as f:

I can't. So in this case, your agonisingly explicit default expression
forces the caller to be *less* explicit when they call the function.

What the right hand giveth, the left hand taketh away.

Can we agree that, like salt, sugar, and bike-shedding, sometimes you
can have *too much* explicitness in code? Sometimes the right level of
explicitness is to have an agreed upon symbol that acts as a short, easy
to read and write, documented and explicit signal to the function "give

me the default value".

And that symbol can be -1, or an enumeration, or a string, or None.

This is not a hack, and its not "implicit".

> None of my comment had anything to do with the caller's perspective.

The caller's perspective is important. The caller has to read the
function signature (either in the documentation, or if they use
`help(function)` in the interpreter, or when their IDE offers up
tooltips or argument completion). The caller has to actually use the
function. We should consider their perspective.

> If my intent is that I want a parameter to default to an empty list, I
> wouldn't assign it to an object called "monkey_pants" by default then add
> code to the body of the function to check if it's "monkey_pants".

Funny you should say that, but *totally by coincidence*, the Bulgarian
word for "default" happens to be transliterated into English as
*monkeypants*, so if I were Bulgarian that is exactly what I might do!
How weird is that?

Nah, I'm kidding of course. But the point I am making is that you are
mocking the concept by using a totally silly example. "monkey_pants"
indeed, ha ha how very droll. But if you used a more meaningful symbol,
like "default", or some enumeration with a meaningful name, or a value
which is a standard idiom such as -1, then the idea isn't so ridiculous.

And None is one such standard idiom.

*Especially* when the default value is not some trivially simple
expression such as an empty list, but a more complicated expression such
as the default buffering used by open files, or in my earlier example of
the default collation (a sequence of letters which depends on the
other input arguments).

> Steven D'Aprano
> "How is it misleading?"
>
> Your way of having a parameter default to an empty list (or whatever) is by
> having the parameter default to None. That's how it's misleading.

All you have done is repeat that it is misleading. I'm still no clearer
why you think it is misleading to document "if the argument is missing
or None, the default behaviour is ...".

Is the documentation false? Does the function not do exactly what it
says it will do if you pass that symbol?

Let's be more concrete.

Do you feel that using -1 for the buffer size in open() is "misleading"?
Do you think that there is ever a possibility that someday Python's file
I/O will use a buffer with *literally* a size of -1?

--
Steve
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/SZVP4KBU4XHMML7GQKNKTILC2LU5QOR2/

[Steven D'Aprano]

On Fri, Dec 03, 2021 at 01:08:50PM +1100, Chris Angelico wrote:

> How, with external calling, are you going to know which name
> references to look up, and where to get their values from?

Isn't the source code available as a string? I will see the names in the
expression.

Or I will disassemble the code object and inspect its byte-code and/or
AST. Or read the documentation. Or maybe the code object will expose
those names in a dunder and I can just look at that.

> This is already a virtually impossible task in Python, which is why
> f-strings cannot be implemented as str.format(**something) for any
> known meaning of "something".

f-strings cannot be implemented as str.format because str.format doesn't
evaluate arbitrary expressions.

>>> f'{1000+len(repr(None))}'
'1004'

> You cannot get your *current* variable set, much less
> the available variables in some other context.

Why not? The default expression object can record nonlocal variables in
a closure, and you can provide your own globals and/or locals.

You're saying "can't, impossible" but I could take your function object,
extract the code object, disassemble the code to AST or byte-code,
extract the portions that represent the default expression, and evaluate
that. So not *impossible*, just bloody inconvenient and annoying.

(Well, when I say *I could*, I mean *somebody with mad leet AST or byte-
code hacking skillz. Not actually me.)

> > > And it is potentially a LOT of unnecessary overhead. Consider this edge case:
> > >
> > > def f(a, b=>c:=len(a)): ...
> > >
> > > In what context should the name c be bound?
> >
> > With late-bound defaults, the expression is evaluated at function call
> > time, in the scope of f()'s locals. So c would be a local.
>
> In other words, if you're trying to evaluate b's default externally,
> you have to set c in a context that doesn't even exist yet. Is that
> correct?

Yes? You seem to be saying that as if it were some fatal flaw in my
proposal.

mylocals = {}

There. Now the context exists.

result = eval(default_expression.__code__, globals(), mylocals())

> > The third obvious answer is that if either the decision or the
> > implementation is really too hard, then make it a syntax error for now,
> > and revisit it in the future.
>
> Everything's perfectly well defined, save that you can't externally
> evaluate the defaults.

You keep saying that, but your arguments are not convincing.

> You can quite happily use the walrus in a
> late-bound default, and it'll bind to the function's locals.

Cool :-)

> The problem is that you're trying to reference cell variables in a
> closure that might not even exist yet, so *just in case* you might
> have nonlocals, you have to construct a closure... or find an existing
> one. Ill-defined and inefficient.

That doesn't make sense to me. When you compile the default expression,
you know what variable names are assigned to with a walrus expression
(if any), you know what names exist in the surrounding nonlocal scope
(if any), you know what names are parameters.

If the default expression doesn't use walrus, doesn't refer to any
nonlocal names, and doesn't refer to any other parameters, why would you
construct a closure "just in case"? What would you put in it?

--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/F3W44ASB5L57TUCTHP55LHVYJ6XZR3SW/

[Steven D'Aprano]

On Fri, Dec 03, 2021 at 10:36:55AM +1100, Chris Angelico wrote:

> Same again. If you consider the equivalent to be a line of code in the
> function body, then the signature has become MASSIVELY more useful.
> Instead of simply seeing "x=<object object at 0x7fba1b318690>", you
> can see "x=>[]" and be able to see what the value would be. It's
> primarily human-readable (although you could eval it),

You: "it is impossible to evaluate the late-bound default outside of the
function!"

Also you: "Just eval the string."

You can't have it both ways :-)


--
Steve

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/ZS6Z7TIJFR2AC2DV7ZVO7HAQX2JJ5NEI/

[Brendan Barnwell]

On 2021-12-02 15:40, Chris Angelico wrote:
> Actually, no. I want to put the default arguments into the signature,
> and the body in the body. The distinction currently has a technical
> restriction that means that, in certain circumstances, what belongs in
> the signature has to be hacked into the body. I'm trying to make it so
> that those can be put where they belong.

Chris, I know this is probably not your intention, but I feel the
discussion is continually muddle by you just saying "default arguments"
as if everyone agrees on what those are and the issue is just where to
put them. But clearly that is not the case. You seem to take it for
granted that "len(x) evaluated when the function is called" "is" a
"default argument" in the same sense that an early-bound default like
the number 3 is. I do not agree, and it's pretty clear David Mertz does
not agree, and I think there are others here who also do not agree.

It's not as is there is some pre-existing notion of "default argument"
in Python and you are just proposing to add an implementation of it that
was left out due to some kind of oversight. Your proposal is CHANGING
the idea of what a default argument is. I get that it's natural to
refer to your new proposal as "default arguments" but I've now seen a
number of messages here where you say "but no we should do X because
this is a default argument", taking for granted that what you're talking
about is already agreed to be a default argument. (No doubt I and many
others are also guilty of similar missteps by saying "default argument"
as a shorthand for something or other, and I'll try to be careful about
it myself.)

By my definition as of now, a default argument has to be an object.
Thus what your proposal envisions are not default arguments. You can't
just say "I want to do this with default arguments". You need to
provide an argument for why we should even consider these to be default
arguments at all.

Perhaps a way of stating this without reference to arguments is this:
you want to put code in the function signature but not have it executed
until the function is called. I do not agree with that choice. The
function signature and the function body are different syntactic
environments with different semantics. Everything in the function
signature should be executed when the function is defined (although of
course that execution can result in an object which contains deferred
behavior itself, like if we pass a function as an argument to another).
Only the function body should be executed when the function is called.
If we want to provide some way to augment the function body to assign
values to missing arguments, I wouldn't rule that out completely, but in
my view the function signature is not an available space for that. The
function signature is ONLY for things that happen when the function is
defined. It is too confusing to have the function signature mix
definition-time and call-time behavior.

--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."
--author unknown

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python...@python.org/message/4B3GTCFZILOLNF6YYTRCW5I3UHVBS3VG/