[Python-ideas] Null coalescing operator

[Arek Bulski]

Sometimes I find myself in need of this nice operator that I used back in the days when I was programming in .NET, essentially an expression 

>>> expr ?? instead

should return expr when it `is not None` and `instead` otherwise. 

A piece of code that I just wrote, you can see a use case:

    def _sizeof(self, context):

        if self.totalsizeof is not None:

            return self.totalsizeof

        else:

            raise SizeofError("cannot calculate size")

With the oprator it would just be

    def _sizeof(self, context):

        return self.totalsizeof ?? raise SizeofError("cannot calculate size")

pozdrawiam,

Arkadiusz Bulski

[Chris Angelico]

On Sat, Sep 10, 2016 at 6:01 AM, Arek Bulski <arek....@gmail.com> wrote:
> Sometimes I find myself in need of this nice operator that I used back in
> the days when I was programming in .NET, essentially an expression
>
>>>> expr ?? instead
>
> should return expr when it `is not None` and `instead` otherwise.

You can use 'or' for this, as long as you're okay with other falsey
values being treated the same way. In a lot of cases, this isn't a
problem.

However, even if this is implemented, it would be in an expression
context, so 'raise' would never work. For that, I'd just use the
explicit statement form.

ChrisA
_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Zachary Ware]

This was proposed almost exactly a year ago, start reading here:
https://mail.python.org/pipermail/python-ideas/2015-September/036289.html

--
Zach

[David Mertz]

This idea has come up before.  While I can see the use of it, to me at least that use doesn't feel nearly common enough to warrant dedicated syntax.

In many cases, it is a "truthy" value you are looking for rather than `is not None` specifically.  That has a convenient spelling:

expr or instead

If it really is the actual None-ness you are curious about, you need the slightly longer:

expr if expr is not None else instead

Your example seems to want to fall back to a statement suite rather than a value.  To do that, you'd have to put the suite inside a function such as:

def Raise(err):

    raise err

And use it something like:

self.totalsizeof or Raise(SizeofError(...))

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--

Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.

[David Mertz]

I'd note you can also save 4 characters by writing:

instead if expr is None else expr

[MRAB]

'raise' is a statement, so it can't appear in an expression.

This has been discussed before, so you might want to read this thread first:

Null coalescing operators
https://mail.python.org/pipermail/python-ideas/2015-September/036289.html

[Steven D'Aprano]

On Fri, Sep 09, 2016 at 10:01:44PM +0200, Arek Bulski wrote:
> Sometimes I find myself in need of this nice operator that I used back in
> the days when I was programming in .NET, essentially an expression
>
> >>> expr ?? instead
>
> should return expr when it `is not None` and `instead` otherwise.

As Zach and MRAB mention, this was discussed last year. If I recall
correctly, the discussion fizzled out without a solid conclusion. I
think there's a PEP -- if not, there should be.

I would be interested in revisiting this idea, but 3.6 feature freeze is
only a day or two away and I won't have time to discuss this before
then. So let's please drop this discussion until the 3.6 beta is
released.


--
Steve

[Ralph Broenink]

It is PEP 505.
I agree we should resume the discussion on this PEP though (for 3.7), I'm not completely sure why it stalled.

Ralph

[Stephen J. Turnbull]

Steven D'Aprano writes:

> As Zach and MRAB mention, this was discussed last year. If I recall
> correctly, the discussion fizzled out without a solid conclusion. I
> think there's a PEP -- if not, there should be.

The problem, as I recall, was that there was a conclusion that
'maybe_None if maybe_None is None else default' and the less precise
'maybe_falsy or default' are sufficient if you just want to provide a
default if a value is "null". The proponents pointed out that other
operators would benefit from null coalescing, and then discussion
petered out because there was no remotely plausible suggestion except
to use '?' to indicate null-coalescing versions. But Guido has
historically resisted any use of '?' whatsoever in Python syntax, and
there weren't any good alternatives or strong proponents of ?-based
syntax.

I forget if Guido was very sympathetic to null-coalescing operators,
given somebody came up with a good syntax.

There was also an issue of whether SQL NULLs and similar constructs
"should" be spelled None in Python, and if not, how would null
coalescence be defined.

[Random832]

On Sat, Sep 10, 2016, at 12:48, Stephen J. Turnbull wrote:
> I forget if Guido was very sympathetic to null-coalescing operators,
> given somebody came up with a good syntax.

As I remember the discussion, I thought he'd more or less conceded on
the use of ? but there was disagreement on how to implement it that
never got resolved. Concerns like, you can't have a?.b return None
because then a?.b() isn't callable, unless you want to use a?.b?() for
this case, or some people wanted to have "a?" [where a is None] return a
magic object whose attribute/call/getitem would give no error, but that
would have to keep returning itself and never actually return None for
chained operators.

[Guido van Rossum]

The way I recall it, we arrived at the perfect syntax (using ?) and
semantics. The issue was purely strong hesitation about whether
sprinkling ? all over your code is too ugly for Python, and in the end
we couldn't get agreement on *that*. Another problem is PEP 505 -- it
is full of discussion but its specification is unreadable due to the
author's idea to defer the actual choice of operators and use a
strange sequence of unicode characters instead.

If someone wants to write a new, *short* PEP that defers to PEP 505
for motivation etc. and just writes up the spec for the syntax and
semantics we'll have a better starting point. IMO the key syntax is
simply one for accessing attributes returning None instead of raising
AttributeError, so that e.g. `foo?.bar?.baz` is roughly equivalent to
`foo.bar.baz if (foo is not None and foo.bar is not None) else None`,
except evaluating foo and foo.bar only once.

--
--Guido van Rossum (python.org/~guido)

[Sven R. Kunze]

On 10.09.2016 19:14, Random832 wrote:
> As I remember the discussion, I thought he'd more or less conceded on
> the use of ? but there was disagreement on how to implement it that
> never got resolved. Concerns like, you can't have a?.b return None
> because then a?.b() isn't callable, unless you want to use a?.b?() for
> this case, or some people wanted to have "a?" [where a is None] return a
> magic object whose attribute/call/getitem would give no error, but that
> would have to keep returning itself and never actually return None for
> chained operators.

That appeared to be one solution to make the ?-syntax class useful.

But in the end, there were too many possibilities (operator?, nomad
objects?, syntax expansion?, something else?), issues with all those "?"
all over the place, hiding errors this way (which was the most serious
one), and uncertainty about the overall benefit of this syntax compared
to better designs like "how to not use None in the first place" didn't
lead to a result so far.


Let's see those can be resolved.


Sven

[Paul Moore]

On 10 September 2016 at 18:26, Guido van Rossum <gu...@python.org> wrote:
> IMO the key syntax is
> simply one for accessing attributes returning None instead of raising
> AttributeError, so that e.g. `foo?.bar?.baz` is roughly equivalent to
> `foo.bar.baz if (foo is not None and foo.bar is not None) else None`,
> except evaluating foo and foo.bar only once.

If we're not looking to use all the other null-coalescing variants
(?=, ?(), ...) - which is something I'm pleased about, as I do think
that scattering that many ?'s about is likely to lead to ugly code -
then it would probably be fine to just use ? for this operation, so
we'd have foo?bar?baz rather than needing foo?.bar?.baz.

Paul

[MRAB]

On 2016-09-10 18:44, Paul Moore wrote:
> On 10 September 2016 at 18:26, Guido van Rossum <gu...@python.org> wrote:
>> IMO the key syntax is
>> simply one for accessing attributes returning None instead of raising
>> AttributeError, so that e.g. `foo?.bar?.baz` is roughly equivalent to
>> `foo.bar.baz if (foo is not None and foo.bar is not None) else None`,
>> except evaluating foo and foo.bar only once.
>
> If we're not looking to use all the other null-coalescing variants
> (?=, ?(), ...) - which is something I'm pleased about, as I do think
> that scattering that many ?'s about is likely to lead to ugly code -
> then it would probably be fine to just use ? for this operation, so
> we'd have foo?bar?baz rather than needing foo?.bar?.baz.
>

I think that's not as clear; the "?." at least looks like a form of
attribute access.

It would also mean that it would be more difficult to add the other
null-coalescing variants later, if the need arose.

[Guido van Rossum]

On Sat, Sep 10, 2016 at 11:09 AM, MRAB <pyt...@mrabarnett.plus.com> wrote:
> On 2016-09-10 18:44, Paul Moore wrote:
>>
>> On 10 September 2016 at 18:26, Guido van Rossum <gu...@python.org> wrote:
>>>
>>> IMO the key syntax is
>>> simply one for accessing attributes returning None instead of raising
>>> AttributeError, so that e.g. `foo?.bar?.baz` is roughly equivalent to
>>> `foo.bar.baz if (foo is not None and foo.bar is not None) else None`,
>>> except evaluating foo and foo.bar only once.
>>
>>
>> If we're not looking to use all the other null-coalescing variants
>> (?=, ?(), ...) - which is something I'm pleased about, as I do think
>> that scattering that many ?'s about is likely to lead to ugly code -
>> then it would probably be fine to just use ? for this operation, so
>> we'd have foo?bar?baz rather than needing foo?.bar?.baz.
>>
> I think that's not as clear; the "?." at least looks like a form of
> attribute access.
>
> It would also mean that it would be more difficult to add the other
> null-coalescing variants later, if the need arose.

Indeed. And ?. is how this is spelled in some other lanuages (C# and Dart).

I forgot one detail that's in PEP 505: e.g. `foo?.bar.baz()` should be
implemented as `foo.bar.baz() if foo is not None else None`. IOW if
foo is None, the entire trailing section `.bar.baz()` should be
skipped. (But this is a property of `?.` as an alternative attribute
access operator; it doesn't mean `?` is a postfix operator on `foo`.)

Another issue already discussed in PEP 505 is a conflict with IPython
(Jupyter Notebook), which uses ? and ?? as custom syntax to request
help. But maybe it can be taught to only recognize those when they're
the last character(s) on the line?

--
--Guido van Rossum (python.org/~guido)

[Alexander Belopolsky]

On Sat, Sep 10, 2016 at 4:56 PM, Guido van Rossum <gu...@python.org> wrote:

Another issue already discussed in PEP 505 is a conflict with IPython
(Jupyter Notebook), which uses ? and ?? as custom syntax to request
help. But maybe it can be taught to only recognize those when they're
the last character(s) on the line?

I think this is already the case:

In [1]: ?foo

Object `foo` not found.

In [2]: foo?

Object `foo` not found.

In [3]: foo?bar

  File "<ipython-input-3-c7de2a05ce6b>", line 1

    foo?bar

       ^

SyntaxError: invalid syntax

[Random832]

On Sat, Sep 10, 2016, at 13:26, Guido van Rossum wrote:
> The way I recall it, we arrived at the perfect syntax (using ?) and
> semantics. The issue was purely strong hesitation about whether
> sprinkling ? all over your code is too ugly for Python

I think that if there's "strong hesitation" about something being "too
ugly" it can't really be described as "the perfect syntax". IIRC there
were a couple alternatives being discussed that would have reduced the
number of question marks to one [or one per object which might be None].

[David Mertz]

I find the '?.' syntax very ugly, much more so in the examples of chained attributes.

A much better way to handle the use case is to wrap objects in a class that gives this "propagating None" behavior with plain attribute access. A nice implementation was presented in this thread.

[Ryan Gonzalez]

https://github.com/kirbyfan64/_frozensafemockobjectimplementation

In all seriousness, though, I really feel like that would be the ultimate bug magnet, since it'd be easy to forget to un-wrap the object afterwards.

--
Ryan
[ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong.
http://kirbyfan64.github.io/

[David Mertz]

Actually, I guess the example I liked was from the year ago discussion.  And it didn't do *exactly* what I think a wrapper should.  What I'd want would be like this:

class NoneCoalesce(object):

    "Standard operations on object for 'is not None'"

    def __init__(self, obj):

        self.obj = obj

    def __getattr__(self, name):

        try:

            return getattr(self.obj, name)

        except AttributeError:

            return NoneCoalesce(None)

    def __getitem__(self, item):

        try:

            return self.obj[item]

        except (TypeError, KeyError):

            return NoneCoalesce(None)

    def __call__(self, *args, **kwds):

        try:

            return self.obj(*args, **kwds)

        except TypeError:

            return NoneCoalesce(None)

    def __bool__(self):

        return self.obj is not None

    def __repr__(self):

        return "NoneCoalesce[%r]" % self.obj

    def __str__(self):

        return "NoneCoalesce[%r]" % self.obj

    def __len__(self):

        try:

            return len(self.obj)

        except TypeError:

            return 0

Then we might use it similar to this:

>>> from boltons.dictutils import OrderedMultiDict

>>> from NoneCoalesce import NoneCoalesce

>>> omd = OrderedMultiDict()

>>> omd['a'] = 1

>>> omd['b'] = 2

>>> omd.add('a', 3)

>>> nc = NoneCoalesce(omd)

>>> nc or "Spanish Inquisition"

Out[8]: NoneCoalesce[OrderedMultiDict([('a', 1), ('b', 2), ('a', 3)])]

>>> nc.spam or "Spam"

Out[9]: 'Spam'

>>> nc['nope'].bar.baz()

Out[10]: NoneCoalesce[None]

>>> nc['a']

Out[11]: 3

>>> nc.getlist('a')

Out[12]: [1, 3]

Nothing special about boltons' OrderedMultiDict here, just something I've been playing with that has some distinctive methods.

The idea is that we can easily have both "regular" behavior and None coalescing just by wrapping any objects in a utility class... and WITHOUT adding ugly syntax.  I might have missed some corners where we would want behavior wrapped, but those shouldn't be that hard to add in principle.

[Guido van Rossum]

So you're offering `NoneCoalesce(x).bar` as less-ugly alternative to
`x?.bar`... Color me unconvinced.

--Guido van Rossum (python.org/~guido)

[Chris Angelico]

On Sun, Sep 11, 2016 at 9:10 AM, Guido van Rossum <gu...@python.org> wrote:
> So you're offering `NoneCoalesce(x).bar` as less-ugly alternative to
> `x?.bar`... Color me unconvinced.

As a syntactic form? Not interested. But what if it's the underlying
implementation? We have "yield from X" as a tidy syntax for roughly a
page of equivalent code. We could have "x?.bar" as syntactic sugar for
"NoneCoalesce(x).bar".

ChrisA

[David Mertz]

On Sat, Sep 10, 2016 at 4:10 PM, Guido van Rossum <gu...@python.org> wrote:

>

> So you're offering `NoneCoalesce(x).bar` as less-ugly alternative to
> `x?.bar`... Color me unconvinced.

No, I'm offering a more realistic use pattern:

> for x in get_stuff():

>
>     x = NoneCoalesce(x)

>
>     # ... bunch of stuff with x ...
>     # ... more stuff with nested keys or attributes ...

>
>     x2 = x.foo

>
>     x3 = x.bar.baz[x2]

>
>     x4 = x(x.val)

>
>     result = x3(x4)

>
>  

As a less ugly alternative in the fairly uncommon case that you want None coalescing as the behavior of getting attributes, keys, call values, etc. that may or may not be available (AND where you don't want to wrap all of those access patterns in one try/except block).

In contrast, the ugly version of even this pretty simple toy code with the hypothetical syntax would be:

> for x in get_stuff():

>
>     # ... bunch of stuff with x ...

>

>     # ... more stuff with nested keys or attributes ...

>
>     x2 = x?.foo

>
>     x3 = x?.bar?.baz?[x2]

>
>     x4 = x?(x?.val)

>
>     result = x3?(x4)

This second case looks absolutely awful to me.  And real world uses, if implemented, would quickly get much worse than that.

[David Mertz]

Sorry, I sent this accidentally as private reply, then tried to fix it on phone.  The latter produced horrible formatting.  Please just read this version.

On Sat, Sep 10, 2016 at 4:10 PM, Guido van Rossum <gu...@python.org> wrote:

So you're offering `NoneCoalesce(x).bar` as less-ugly alternative to
`x?.bar`... Color me unconvinced.

No, I'm offering a more realistic use pattern:

for x in get_stuff():

    x = NoneCoalesce(x)

    # ... bunch of stuff with x ...

    # ... more stuff with nested keys or attributes ...

    x2 = x.foo

    x3 = x.bar.baz[x2]

    x4 = x(x.val)

    result = x3(x4)

 

As a less ugly alternative in the fairly uncommon case that you want None coalescing as the behavior of getting attributes, keys, call values, etc. that may or may not be available (AND where you don't want to wrap all of those access patterns in one try/except block).

In contrast, the ugly version of even this pretty simple toy code with the hypothetical syntax would be:

for x in get_stuff():

    # ... bunch of stuff with x ...

    # ... more stuff with nested keys or attributes ...

    x2 = x?.foo

    x3 = x?.bar?.baz?[x2]

    x4 = x?(x?.val)

    result = x3?(x4)

This second case looks absolutely awful to me.  And real world uses, if implemented, would quickly get much worse than that.

Yours, David...

--

Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.

[Guido van Rossum]

On Sat, Sep 10, 2016 at 4:27 PM, Chris Angelico <ros...@gmail.com> wrote:
> On Sun, Sep 11, 2016 at 9:10 AM, Guido van Rossum <gu...@python.org> wrote:
>> So you're offering `NoneCoalesce(x).bar` as less-ugly alternative to
>> `x?.bar`... Color me unconvinced.
>
> As a syntactic form? Not interested. But what if it's the underlying
> implementation? We have "yield from X" as a tidy syntax for roughly a
> page of equivalent code. We could have "x?.bar" as syntactic sugar for
> "NoneCoalesce(x).bar".

PEP 505 has an option for a way to customize the coalescing operation
(https://www.python.org/dev/peps/pep-0505/#generalized-coalescing).
Though I think I'd rather not do that.

But it just occurs to me that the implementation given by David Mertz
is not what I'd expect: it seems that `NoneCoalesce([]).flup` would
catch the AttributeError (there' no `[].flup`) and return
NoneCoalesce(None), whereas I would expect `?.` to only return None
when the LHS is None, not when some other not-None object doesn't have
the requested attribute. (And the "pile of poo" operator in PEP 505
agrees with me.)

--
--Guido van Rossum (python.org/~guido)

[Guido van Rossum]

There seems to be a major misunderstanding here. A None-coalescing
operator is not for catching AttributeError, it's a shortcut similar
to "a or b" except that it checks for "a is None" rather than bool(a).

[David Mertz]

On Sep 10, 2016 4:45 PM, "Guido van Rossum" <gu...@python.org> wrote:
>
> There seems to be a major misunderstanding here. A None-coalescing
> operator is not for catching AttributeError, it's a shortcut similar
> to "a or b" except that it checks for "a is None" rather than bool(a).

That's exactly what the wrapper does. Except it converts all the regular operators into their None-coalescing versions by putting the extra checks into the wrapped object itself.

Now admittedly, this DOES mean that the behavior of operations is somewhat different depending on whether they are wrapped objects or not. False-like is a different thing than None-like.

This really MUST BE essentially a way a catching AttributeErrors though. With the proposed syntax 'x?.foo?.bar' will resolve even if x has no 'foo'. So 'x?.'foo' has to be something special. I guess that special thing could be a 3.7-style None that responds to new operators, but that's essentially still *wrapping* a 3.6-style None.

In my mind, as I say, the question marks look ugly, especially when repeated in chained operations (attribute, call, item get). But even if they didn't feel bad visually, I don't believe the use case is common enough to warrant dedicated syntax. Even if my keyboard had some character I thought was beautiful and intuitive for that meaning, it's still an extra cognitive burden to distinguish the plain from None-coalescing versions of every operation, especially for learners.

Another problem is that the question mark still doesn't actually get the special 'a or b' behavior. For that you still need 'a if a is not None else b'. Or I guess, in concept, 'a ?or b'. For what it's worth, the wrapper gives you the special 'a or b' semantics by casting non-Nones as truthy... But again, 'a' has to have been wrapped first.

[Guido van Rossum]

No. PEP 505 actually solves the problem without ever catching
AttributeError. Please read it.

[David Mertz]

On Sat, Sep 10, 2016 at 5:23 PM, Guido van Rossum <gu...@python.org> wrote:

No. PEP 505 actually solves the problem without ever catching
AttributeError. Please read it.

I read it again (I did a year ago, but reviewed it now).  I hadn't been thinking that the *mechanism* of a new None-coalescing operator would actually be catching an exception.  It could (and should) work differently if it becomes syntax.  

What I was getting at with "essentially" was that it would *do the same thing* that an AttributeError does.  That is, if `x.foo` can't be evaluated (i.e. x doesn't have an attribute 'foo'), then access is informally "an error."  The hypothetical "x?.foo" catches that "error" and substitutes a different value.  The particular implementation under-the-hood is less important for most programmers who might use the construct (and I think documentation would actually give an informal equivalent as something similar to what I put in the NoneCoalesce class).

[MRAB]

On 2016-09-11 02:02, David Mertz wrote:
> On Sat, Sep 10, 2016 at 5:23 PM, Guido van Rossum <gu...@python.org
> <mailto:gu...@python.org>> wrote:
>
> No. PEP 505 actually solves the problem without ever catching
> AttributeError. Please read it.
>
>
> I read it again (I did a year ago, but reviewed it now). I hadn't been
> thinking that the *mechanism* of a new None-coalescing operator would
> actually be catching an exception. It could (and should) work
> differently if it becomes syntax.
>
> What I was getting at with "essentially" was that it would *do the same
> thing* that an AttributeError does. That is, if `x.foo` can't be
> evaluated (i.e. x doesn't have an attribute 'foo'), then access is
> informally "an error." The hypothetical "x?.foo" catches that "error"
> and substitutes a different value. The particular implementation
> under-the-hood is less important for most programmers who might use the
> construct (and I think documentation would actually give an informal
> equivalent as something similar to what I put in the NoneCoalesce class).
>

x?.foo would lookup attribute 'foo' _unless_ x was None, in which case
it would return None. It's simply:

None if x is None else x.foo

This means that None?.__str__() would return None, not 'None'. (None has
an attribute called '__str__', and None.__str__() returns 'None', but it
would not be looked up because None is, well, None.)

[David Mertz]

Ok, I have been thinking of the behavior too broadly. I realize now that `x?.foo` might still simply raise an AttributeError if x is neither None nor a thing with a foo attribute.

The class I wrote is definitely too aggressive for the behavior described. On the other hand, by being narrower in behavior there feels like even less motivation for new syntax.

[Guido van Rossum]

To the contrary. I read and write code that performs explicit checks
for None all the time. Catching AttributeError is often a code smell
or at least a measure of last resort.

--
--Guido van Rossum (python.org/~guido)

[David Mertz]

How much of the time is a branch of the None check a single fallback value or attribute access versus how often a suite of statements within the not-None branch?

I definitely check for None very often also. I'm curious what the breakdown is in code I work with.

[Random832]

On Sat, Sep 10, 2016, at 19:38, David Mertz wrote:
> x2 = x?.foo
>
> x3 = x?.bar?.baz?[x2]

A. if you're doing three different things with x, why are you using this
instead of wrapping it in an if statement?

B. Under some of the proposals, unless x.bar might be None independently
of x not being None, this would just be "x?.bar.baz[x2]

> x4 = x?(x?.val)

C. Under some of the proposals, the inner contents of the call brackets
aren't evaluated if x is None, so you don't need to use x?.val here.

> result = x3?(x4)

[Random832]

On Sat, Sep 10, 2016, at 20:15, David Mertz wrote:
> On Sep 10, 2016 4:45 PM, "Guido van Rossum" <gu...@python.org> wrote:
> >
> > There seems to be a major misunderstanding here. A None-coalescing
> > operator is not for catching AttributeError, it's a shortcut similar
> > to "a or b" except that it checks for "a is None" rather than bool(a).

To put it more explicitly, more similar to "a and a.b"

> This really MUST BE essentially a way a catching AttributeErrors though.
> With the proposed syntax 'x?.foo?.bar' will resolve even if x has no
> 'foo'.

Why? I think you're confusing this proposal for something else. This is
for treating x is None specially, not simply catching AttributeError to
deal with the fact that None.foo doesn't exist.

[David Mertz]

For kicks I looked at the code in a commercial product that we are open sourcing very soon at Continuum Analytics.  It has about 27k lines of Python and Cython.  I checked it just by running `grep -C3 'if.*None'` over the source code and eyeballing.

Obviously, this code might be refactored if other syntax was available.  And I personally wrote only a small part of the code, so I might write it differently either way.  But looking through, the uses of 'is (not) None' I found fell into a few categories.  

Some were suites of statements under the 'foo is None' branch.  Those would have to remain suites.  Some are single line returns, other single line raise.  Those I call 'value' are the ones that would lend themselves to the 'bar = foo ?? baz' style.  The ones I call 'attribute' are the cases where we test for non-None before accessing an attribute of the object.

The thing that I found surprisingly common was cases where a compound condition was used, e.g. 'if x is None and y > 37'.  Those might be possible to refactor, but cannot be directly rewritten in the coalescing style.

  - Suite: 75

  - Return: 21

  - Raise: 13

  - Value: 46

  - Attribute: 2

  - Compound condition: 25

[Bruce Leban]

On Sat, Sep 10, 2016 at 6:02 PM, David Mertz <me...@gnosis.cx> wrote:

What I was getting at with "essentially" was that it would *do the same thing* that an AttributeError does.  That is, if `x.foo` can't be evaluated (i.e. x doesn't have an attribute 'foo'), then access is informally "an error."  The hypothetical "x?.foo" catches that "error" and substitutes a different value.  The particular implementation under-the-hood is less important for most programmers who might use the construct (and I think documentation would actually give an informal equivalent as something similar to what I put in the NoneCoalesce class).

That's not a good way to think about it. This new operator is only checking for None and not actually checking for AttributeErrors. Consider:

(3).x    # AttributeError

{}.x     # AttributeError

None.x   # AttributeError

(3)?.x   # still AttributeError

{}?.x    # still AttributeError

None?.x  # None

And also:

None.__class__   # <type 'NoneType'>

None?.__class__  # None

And it's certainly not the case that those values don't accept any attributes: 

(3).real        # 3

{}.values       # <built-in method ...>

None.__class__  #<type 'NoneType'>

--- Bruce

Check out my puzzle book and get it free here:

http://J.mp/ingToConclusionsFree (available on iOS)

[Daniel Moisset]

Both this discussion, PEP 505, and the one a year ago, tend to mix up 2 related but separate proposals:

w

(A) Add a None-coalescing operator (like C# a ?? b, what you would write in Python as "a or b" if it didn't have the falsy gotcha)

(B) Add some None-aware navigation operators ( The "?.", "?()", "?[]", or what you would write in python as "a and a.attribute" if it didn't have the falsy gotcha)

Both are things that can be already done in python, so the purpose here is to add some convenience (aka "syntax sugar"). IMO, this kind of syntax sugar proposals should be weighed with the frequency of the coding pattern where the sugar can be applied. And from the stats presented in PEP-505 (B) is one order of magnitude less usual than (A); that matches most of the examples I see in the threads and FWIW my personal experience.

So, as a counterproposal I would like to suggest:

* Add an "a ?? b" operator which is equivalent to "a if a is None else b" (but evaluating a once)

* Do not add none-aware navigation; in the less usual scenario where you need to do it AND ALSO the "and" operator is not usable (it frequently is, given that by default classes are truish), well, you can use a ternary operator

* I don't care if it's an alternate syntax (I'm surprised nobody suggested "||" which is also used in other languages for similar purpose)

Would this satisfy most of the people requesting this? (and, would it satisfy the people making the decision?)

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--

Daniel F. Moisset - UK Country Manager

www.machinalis.com

Skype: @dmoisset

[Guido van Rossum]

On Sun, Sep 11, 2016 at 12:44 PM, Daniel Moisset
<dmoi...@machinalis.com> wrote:
> Both this discussion, PEP 505, and the one a year ago, tend to mix up 2
> related but separate proposals:

I don't think there's that much of a mix-up. PEP 505 clearly describes
each proposal separately and even gives a choice to accept or reject
each one separately.

> (A) Add a None-coalescing operator (like C# a ?? b, what you would write in
> Python as "a or b" if it didn't have the falsy gotcha)

https://www.python.org/dev/peps/pep-0505/#none-coalescing-operator

> (B) Add some None-aware navigation operators ( The "?.", "?()", "?[]", or
> what you would write in python as "a and a.attribute" if it didn't have the
> falsy gotcha)

https://www.python.org/dev/peps/pep-0505/#none-aware-attribute-access-operator

> Both are things that can be already done in python, so the purpose here is
> to add some convenience (aka "syntax sugar"). IMO, this kind of syntax sugar
> proposals should be weighed with the frequency of the coding pattern where
> the sugar can be applied. And from the stats presented in PEP-505 (B) is one
> order of magnitude less usual than (A); that matches most of the examples I
> see in the threads and FWIW my personal experience.

I can't argue here yet. Honestly I looked at some examples; for those
where it wasn't instantly clear that a None-coalescing operator would
*not* help, I found it hard to figure out how to rewrite it. E.g. this
one -- quick, is there a better way?

return "Illegal Argument" + (self.message is not None and (": " +
self.message) or "")

I think the answer is, if we had both None-coalescing (??) and
None-severing (!!) it could be written as follows:

return "Illegal Argument" + ((self.message !! (": " + self.message)) ?? "")

but it took me way too long to prove that to myself.

> So, as a counterproposal I would like to suggest:
>
> * Add an "a ?? b" operator which is equivalent to "a if a is None else b"
> (but evaluating a once)

> * Do not add none-aware navigation; in the less usual scenario where you
> need to do it AND ALSO the "and" operator is not usable (it frequently is,
> given that by default classes are truish), well, you can use a ternary
> operator

Honestly the one thing that makes `?.` attractive is that it's easier
than the None-coalescing and -severing operators to grasp at an
intuitive level. If "foo.bar" raises "AttributeError: 'NoneType'
object has no attribute 'foo'" then try again with "foo?.bar". It's
surprising how often that will work!

> * I don't care if it's an alternate syntax (I'm surprised nobody suggested
> "||" which is also used in other languages for similar purpose)

Interestingly, after analyzing the above example I desperately want to
write it as

return "Illegal Argument" + (self.message && (": " + self.message) || "")

Note that I already know the relative priorities of && and ||, so I
can drop a set of parentheses.

> Would this satisfy most of the people requesting this? (and, would it
> satisfy the people making the decision?)

Personally, after the above example, I'm less excited about ??/!! or
||/&&, and more excited about `?.` -- so it doesn't satisfy me.

--
--Guido van Rossum (python.org/~guido)

[David Mertz]

On Sun, Sep 11, 2016 at 12:44 PM, Daniel Moisset <dmoi...@machinalis.com> wrote:

Both this discussion, PEP 505, and the one a year ago, tend to mix up 2 related but separate proposals:

w

(A) Add a None-coalescing operator (like C# a ?? b, what you would write in Python as "a or b" if it didn't have the falsy gotcha)

(B) Add some None-aware navigation operators ( The "?.", "?()", "?[]", or what you would write in python as "a and a.attribute" if it didn't have the falsy gotcha)

I readily confess that my initial comments here had a think-o about just what was being discussed, notwithstanding having also followed the discussion a year ago.  I somehow had in mind that "None-coalescing" meant something like "get to None on failure" (i.e. *coalesce* to)

I think the reason I thought wrong was because the second set of syntax really encourages that wrong way of thinking.  I don't find `a ?? b` particularly ugly, even if I don't entirely want it.  Similarly if it were spelled `a || b`.  Both of those feel easy to conceptualize and teach as "another kind of short-circuiting, similar to boolean operators."  Maybe an actual word is better? `a ifNone b`? `a failto b`?

Falsey is similar to None-like (I know it's actually None-identical), so the shortcut idea is familiar.  So I'm only -0 on that idea.

However, the None-aware navigation operators suggest something very different.  `a?.foo` MIGHT BE an attribute access, or it might not be.  Likewise, `a?[x]` might be an item get or it might not be.  And `a?(y)` might or might not be a function call.  Obviously, if some or all of those forms are not added they are simply syntax errors.

What operation happens when we use these syntax forms is "it depends"... that answer feels less straightforward than "We call `a.__getattr__('foo')`" (or `a.__getitem__(x)` or `a.__call__(y)`, as the case may be).  Even if there were some characters I found attractive for these "it depends" operations, that would introduce a needless non-uniformity into the semantics of Python.

 

In contrast, a fully spelled out ternary doesn't give me this uneasiness.  Of course the following expression might be one thing or the other, but that's more obvious in the ternary syntax (also more general, you can put *any* values on both branches of the ternary):

    None if a is None else a.foo

[MRAB]

I think this is better:

return "Illegal Argument" + ("" if self.message is None else ": " +
self.message)

but it doesn't require a None-coalescing operator.

[Guido van Rossum]

On Sun, Sep 11, 2016 at 6:00 PM, David Mertz <me...@gnosis.cx> wrote:
> None if a is None else a.foo

This is the crux of the matter to me. It's just too verbose, and the
`if` and `else` keywords are lost in the noise of all the other words
on the line. Plus the big win when it applies) is that if `a` is in
fact something more complex, like `f(a)`, repeating it twice sounds
like a performance penalty, and that's where `f(a)?.foo` really
shines.

--
--Guido van Rossum (python.org/~guido)

[David Mertz]

On Sun, Sep 11, 2016 at 6:11 PM, Guido van Rossum <gu...@python.org> wrote:

On Sun, Sep 11, 2016 at 6:00 PM, David Mertz <me...@gnosis.cx> wrote:
>     None if a is None else a.foo

This is the crux of the matter to me. It's just too verbose, and the
`if` and `else` keywords are lost in the noise of all the other words
on the line. Plus the big win when it applies) is that if `a` is in
fact something more complex, like `f(a)`, repeating it twice sounds
like a performance penalty, and that's where `f(a)?.foo` really
shines.

The non-repetition is certain a plus, I readily confess.  It's not only performance even; `f()` might not be a pure function. Silly example:

>>> def f(a):

....     if random() < .01:

....         return None

....     class V: pass

....     v = V()

....     v.foo = random()*a

....     return v

[Michel Desmoulin]

I messed up my answer and replied to one person instead of the list, so
I'll post it again.

There is also an alternative to this operator, and it's allowing a
shortcut to do:

try:
val = do_thing()
except ThingError:
val = "default"

In the form of:

val = do_thing() except ThingError: "default"

I was debated, and rejected, but I feel like mentioning it again because
it has some strong benefits.

First, it handles the null coalescing very quite well:

val = obj.foo.bar.hey except AttributeError: "default"

But it also can deal with many common operations in Python without the
need to add more operators or variants:

val = my_list[0] except IndexError: "default"

val = iterable[0] except TypeError: next(iter(iterable))

val = int(param) except ValueError: man.nan

It's quite readable, in the same vein of val = foo if bar else
"default", but also familiar since it's using known keyword. And it
doesn't require to add a new operator support in the parser.

Another serious benefits is that it fits current use cases, AND futur
use cases. Indeed, since it leverages Python exception mechanism, any
lib implementing a clean error model can immediately let the users
benefit from it without having to implement, tests and document lots of
helper methods with "default" keywords and the like, while not being
limited to a set of values some operators would only care about, such as
None.

Plus, since EAFP is a popular and handy pattern, it makes sense.

At last, it has the same characteristic as the null coalescing operator:
it's lazy, and hence has a small performance interest too compared to
functional equivalent.

Did I mention it's also easy to expend to a full try/except when you
need something more complicated ? And then you benefit from else and
finally immediately.

I already have many code that would benefit from such a syntax, and I'd
like to hear again what you think about it.

> Python...@python.org <mailto:Python...@python.org>
> https://mail.python.org/mailman/listinfo/python-ideas

> <https://mail.python.org/mailman/listinfo/python-ideas>
> Code of Conduct: http://python.org/psf/codeofconduct/
> <http://python.org/psf/codeofconduct/>
>
>
>
>
> --
> Daniel F. Moisset - UK Country Manager

> www.machinalis.com <http://www.machinalis.com>
> Skype: @dmoisset

[Ethan Furman]

On 09/12/2016 12:05 AM, Michel Desmoulin wrote:

> There is also an alternative to this operator, and it's allowing a
> shortcut to do:
>
> try:
> val = do_thing()
> except ThingError:
> val = "default"
>
> In the form of:
>
> val = do_thing() except ThingError: "default"
>
> I was debated, and rejected, but I feel like mentioning it again because
> it has some strong benefits.

+1

There are many places in my code where this would clean things up a bit. Not having it is like not having list comps or not having the ternary if-else -- possible, but it would be much nicer to have it.

--
~Ethan~

[Ivan Levkivskyi]

On 12 September 2016 at 09:05, Michel Desmoulin <desmoul...@gmail.com> wrote:

In the form of:

val = do_thing() except ThingError: "default"

[...]

But it also can deal with many common operations in Python without the
need to add more operators or variants:

val = my_list[0] except IndexError: "default"

val = iterable[0] except TypeError: next(iter(iterable))

val = int(param) except ValueError: man.nan

 I like this idea, I would propose a (maybe crazy) addition to it. What about a special exception NoneError, that will catch TypeError, AttributeError etc. but only when it was caused by None(),

None.attr, None[1], etc. With this one can write:

x = a.b()[0] except NoneError: 'default'

without a risk of catching other (unrelated) exceptions.

--

Ivan

[Rob Cliffe]

+1, you're preaching to the converted.
This is PEP 463, "Exception-catching expressions".
Except that the PEP required parentheses around a whole exception-catching expression, for reasons that are not clear to me, i.e.
    val = (my_list[0] except IndexError: "default")
rather than

    val = my_list[0] except IndexError: "default"

Rob Cliffe

[Random832]

On Mon, Sep 12, 2016, at 06:01, Ivan Levkivskyi wrote:
> I like this idea, I would propose a (maybe crazy) addition to it. What
> about a special exception NoneError, that will catch TypeError,
> AttributeError etc. but only when it was caused by None(),
> None.attr, None[1], etc. With this one can write:
>
> x = a.b()[0] except NoneError: 'default'
>
> without a risk of catching other (unrelated) exceptions.

Unless there's another None somewhere inside b or b().__getitem__.

[Rob Cliffe]

Assuming you can't break existing code that already traps TypeError, AttributeError, etc., I don't see how you can do this without
having separated kinds of NoneError which were subclasses of TypeError, AttributeError, etc.
Rob Cliffe

[Chris Angelico]

On Tue, Sep 13, 2016 at 12:03 AM, Rob Cliffe <rob.c...@btinternet.com> wrote:
> Assuming you can't break existing code that already traps TypeError,
> AttributeError, etc., I don't see how you can do this without
> having separated kinds of NoneError which were subclasses of TypeError,
> AttributeError, etc.

class NoneError(Exception): pass
class TypeNoneError(TypeError, NoneError): pass
class AttributeNoneError(AttributeError, NoneError): pass

Now you can catch NoneError to catch None.xyz, or AttributeError to
catch foo.xyz for any foo.

I don't think it's good, but it's possible.

ChrisA

[Guido van Rossum]

For the record, I still really don't like PEP 463. We should strive to
catch fewer exceptions, not make it easier to catch them.

--
--Guido van Rossum (python.org/~guido)

[Eric Snow]

On Mon, Sep 12, 2016 at 1:05 AM, Michel Desmoulin
<desmoul...@gmail.com> wrote:
> There is also an alternative to this operator, and it's allowing a
> shortcut to do:
>
> try:
> val = do_thing()
> except ThingError:
> val = "default"
>
> In the form of:
>
> val = do_thing() except ThingError: "default"

Note that there's a subtle difference here when multiple lookups are
involved. Given:

def f(spam):
return spam().eggs().ham

With null-coalescing:

def f(spam):
return spam()?.eggs()?.ham

This is roughly equivalent to:

def f(spam):
_spam = spam()
try:
eggs = _spam.eggs
except AttributeError:
return None
_eggs = eggs()
try:
return _eggs.ham
except AttributeError:
return None

With PEP 463 it doesn't work out so well. The "obvious" spelling would be:

def f(spam):
return (spam().eggs().ham except AttributeError: None)

This is roughly equivalent to:

def f(spam):
try:
return spam().eggs().ham
except AttributeError:
return None

Note how it's different. For one thing, it could mask AttributeError
coming from the calls. For another, you no longer explicitly identify
which lookups to handle. I would expect both to lead to subtle bugs,
whereas with null-coalescing you don't have those problems.

-eric

[Paul Moore]

On 12 September 2016 at 21:47, Eric Snow <ericsnow...@gmail.com> wrote:
> Note that there's a subtle difference here when multiple lookups are
> involved. Given:
>
> def f(spam):
> return spam().eggs().ham
>
> With null-coalescing:
>
> def f(spam):
> return spam()?.eggs()?.ham
>
> This is roughly equivalent to:
>
> def f(spam):
> _spam = spam()
> try:
> eggs = _spam.eggs
> except AttributeError:
> return None
> _eggs = eggs()
> try:
> return _eggs.ham
> except AttributeError:
> return None

From previous explanations in this thread, that's *not* the behaviour at all.

If I understand the proposal, f is actually intended to be equivalent to:

def f(spam):
spam_val = spam()
if spam_val is None:
return None
eggs_val = spam_val.eggs()
if eggs_val is None:
return None
return eggs_val.ham

Personally, I find it pretty worrying that there's this persistent
confusion over what the proposed syntax means. Sure, it's explained in
the PEP and proposal, but if it gets implemented it'll be in the docs.
And yet people don't seem to read any of those - and their intuition
of what the construct does is wrong.

IMO, the syntax may well be useful, but if we don't address the
problem that what people *think* it means isn't what it actually
means, then it'll do more harm than good.

Paul

[Ethan Furman]

On 09/12/2016 02:13 PM, Paul Moore wrote:

Perhaps the bumper-sticker explanation is:

None-coalescing is not to /recover/ from an AttributeError, but to _prevent it_ in the first place -- but only when the base object is already None.

Okay, several bumper stickers. ;)

--
~Ethan~

[Eric Snow]

On Mon, Sep 12, 2016 at 3:13 PM, Paul Moore <p.f....@gmail.com> wrote:
> From previous explanations in this thread, that's *not* the behaviour at all.

Gah, you're right. I knew better too. The diversion back to PEP 463
crossed some wires in my brain. :)

>
> If I understand the proposal, f is actually intended to be equivalent to:
>
> def f(spam):
> spam_val = spam()
> if spam_val is None:
> return None
> eggs_val = spam_val.eggs()
> if eggs_val is None:
> return None
> return eggs_val.ham

...which is even less related to PEP 463.

>
> Personally, I find it pretty worrying that there's this persistent
> confusion over what the proposed syntax means. Sure, it's explained in
> the PEP and proposal, but if it gets implemented it'll be in the docs.
> And yet people don't seem to read any of those - and their intuition
> of what the construct does is wrong.

The tricky bit is the syntactic association with attribute lookup.
However, I don't think it's quite so confusing. As noted above, my
confusion was mostly due to the recent mention of PEP 463.

-eric

[Rob Cliffe]

On 12/09/2016 16:37, Guido van Rossum wrote:
> For the record, I still really don't like PEP 463. We should strive to
> catch fewer exceptions, not make it easier to catch them.

Can you please clarify what you are saying in the last sentence?
The first time I read it, my brain parsed it as "People should raise
fewer exceptions, i.e. use LBYL less".
Later it seemed more likely that you meant "The set of exceptions we
catch in a specific piece of code should not be too large; we should
only catch those exceptions that we really mean to handle".
But perhaps you mean something else altogether?

Also I don't see the connection between the first sentence and the
second. Exception-catching expressions as per PEP 463 just give you a
more concise way of of doing something you can do with try+except. Do
you mean it would make it easier to catch exceptions because you have to
type fewer characters, and that this would be a bad thing?
Thanks,
Rob Cliffe

[Ethan Furman]

On 09/12/2016 08:37 AM, Guido van Rossum wrote:

> For the record, I still really don't like PEP 463. We should strive to
> catch fewer exceptions, not make it easier to catch them.

I certainly agree with the first part, slightly reworded: we should strive to generate fewer exceptions that we have to catch.

I disagree with the second part: being able to condense four lines of code (1 for try, 1 for except, 1 for the attempt, and 1 for recovery) in to one line of code seems like a win. I know I find it frustrating when my choice is between the 4-line boiler-plate try/except, or an equally verbose and ugly multi-line non-exception generating stanza.

Anyway, my two cents worth. If you collect them all for this subject I think I owe you a dime. ;)

--
~Ethan~

[Rob Cliffe]

On 13/09/2016 01:45, Rob Cliffe wrote:
>
>
> On 12/09/2016 16:37, Guido van Rossum wrote:
>> For the record, I still really don't like PEP 463. We should strive to
>> catch fewer exceptions, not make it easier to catch them.
> Can you please clarify what you are saying in the last sentence?
> The first time I read it, my brain parsed it as "People should raise
> fewer exceptions, i.e. use LBYL less".

Typo: I meant EAFP not LBYL - sorry!

[Guido van Rossum]

On Mon, Sep 12, 2016 at 5:45 PM, Rob Cliffe <rob.c...@btinternet.com> wrote:
>
>
> On 12/09/2016 16:37, Guido van Rossum wrote:
>>
>> For the record, I still really don't like PEP 463. We should strive to
>> catch fewer exceptions, not make it easier to catch them.
>
> Can you please clarify what you are saying in the last sentence?
> The first time I read it, my brain parsed it as "People should raise fewer
> exceptions, i.e. use LBYL less".

(If that means "Leap Before You Look", yes. :-)

> Later it seemed more likely that you meant "The set of exceptions we catch
> in a specific piece of code should not be too large; we should only catch
> those exceptions that we really mean to handle".
> But perhaps you mean something else altogether?
>
> Also I don't see the connection between the first sentence and the second.
> Exception-catching expressions as per PEP 463 just give you a more concise
> way of of doing something you can do with try+except. Do you mean it would
> make it easier to catch exceptions because you have to type fewer
> characters, and that this would be a bad thing?

Yeah, that's exactly my point. PEP 463 gives you a shorter way to
catch an exception, so it gives you less motivation to find a way to
write your code (or define your API) that doesn't involve catching
exceptions. But APIs involving exceptions are often inferior to APIs
that don't require exception catching. (Yes, I am aware of __next__()
raising StopIteration -- but that API design usually doesn't require
you to catch it.)

--
--Guido van Rossum (python.org/~guido)

[Nick Coghlan]

On 13 September 2016 at 07:13, Paul Moore <p.f....@gmail.com> wrote:
> If I understand the proposal, f is actually intended to be equivalent to:
>
> def f(spam):
> spam_val = spam()
> if spam_val is None:
> return None
> eggs_val = spam_val.eggs()
> if eggs_val is None:
> return None
> return eggs_val.ham
>
> Personally, I find it pretty worrying that there's this persistent
> confusion over what the proposed syntax means. Sure, it's explained in
> the PEP and proposal, but if it gets implemented it'll be in the docs.
> And yet people don't seem to read any of those - and their intuition
> of what the construct does is wrong.

Right, there are two quite reasonable interpretations for what a
conditional attribute lookup does, and the "catch and ignore
AttributeError" case is going to be more familiar to most Pythonistas,
as that's the way getattr() works when given a default value.

Consider the case of chained attribute lookup as a function using a
"None" default to getattr():

def attr_chain_abort_on_missing(base, *attrs):
result = base
for attr in attrs:
if result is None:
break
result = getattr(result, attr, None)
return result

vs the actually proposed semantics:

def attr_chain_abort_only on_none(base, *attrs):
result = base
for attr in attrs:
if result is None:
break
result = getattr(result, attr)
return result

In the latter version, AttibuteError escapes - it's only when the
original value is None, or an attribute exists and is None that the
iteration bails out early without raising an exception.

> IMO, the syntax may well be useful, but if we don't address the
> problem that what people *think* it means isn't what it actually
> means, then it'll do more harm than good.

Agreed, and I think a key indicator for that would be whether or not
people that saw:

result = obj?.first?.second?.third

agreed on whether or not it could raise AttributeError or TypeError.

Any prospective PhD students in the audience looking for a language
usability question to study before 3.7b1 rolls around in 2018? :)

Cheers,
Nick.

P.S. I'll note that the *upside* I see to (this part of) the proposal
is that it would implement a *very* fine-grained "is not None" check,
where the alternative in real code would often be an overly broad
exception handling clause, like:

try:
result = obj.first().second.third
except (TypeError, AttributeError):
return None

rather then actually spelling out the fine-grained checks properly.

--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia

[Rob Cliffe]

On 13/09/2016 04:43, Guido van Rossum wrote:
> On Mon, Sep 12, 2016 at 5:45 PM, Rob Cliffe <rob.c...@btinternet.com> wrote:
>>
>> On 12/09/2016 16:37, Guido van Rossum wrote:
>>> For the record, I still really don't like PEP 463. We should strive to
>>> catch fewer exceptions, not make it easier to catch them.
>> Can you please clarify what you are saying in the last sentence?
>> The first time I read it, my brain parsed it as "People should raise fewer
>> exceptions, i.e. use LBYL less".
> (If that means "Leap Before You Look", yes. :-)

I meant EAFP, but ! like your version! :-) Sorry for the confusion.

>
>> Later it seemed more likely that you meant "The set of exceptions we catch
>> in a specific piece of code should not be too large; we should only catch
>> those exceptions that we really mean to handle".
>> But perhaps you mean something else altogether?
>>
>> Also I don't see the connection between the first sentence and the second.
>> Exception-catching expressions as per PEP 463 just give you a more concise
>> way of of doing something you can do with try+except. Do you mean it would
>> make it easier to catch exceptions because you have to type fewer
>> characters, and that this would be a bad thing?
> Yeah, that's exactly my point. PEP 463 gives you a shorter way to
> catch an exception, so it gives you less motivation to find a way to
> write your code (or define your API) that doesn't involve catching
> exceptions. But APIs involving exceptions are often inferior to APIs
> that don't require exception catching. (Yes, I am aware of __next__()
> raising StopIteration -- but that API design usually doesn't require
> you to catch it.)
>

You surprise me. I thought LBYL and EAFP were both approved Python
idioms, in some cases one being better, in some cases another, choice to
be made on the merits of each case (or the author's preference). I
certainly use both (and sometimes time both to see which is faster).
Now it sounds as if you're trying to impose a style guide on the world
by discouraging the EAFP. And wasn't the discussion general, not about
APIs specifically?
Best wishes,
Rob Cliffe

[Nick Coghlan]

Which is preferable depends greatly on context of use, which is why
you'll find a lot of Python APIs offer both forms - it's not *just* a
matter of inheriting the exceptionless version from C, and then later
adding a Python version that gives an exception instead of None or a
user-supplied default value.

It's similar to why IEEE754 defines both quiet NaN *and* signalling
NaN - which one you want depends on what you're doing.

In web servers, for example, you'll often have lots of fields where
"not present" is a perfectly acceptable return value. For those, APIs
that just return None for unknown entries are very handy, which is why
SQL Alchemy offers both ".first()" and "one()", which mainly differ in
how and when they throw an exception, rather than what they do when
they succeed.

However, blindly catching *all* exceptions from a complex
subexpression is rarely the right thing to do, so APIs that only offer
"this may throw exceptions during normal operation under these
circumstances" without a convenience wrapper that does the exception
handling for you can end up being a pain to work with.

PEP 463 makes those APIs less painful to deal with, but at the cost of
encouraging overly broad exception handlers. By contrast, fixing APIs
on a case-by-case basis puts the design burden where it can do the
most good: on the API designer, who can make the scope of the
exception handling suitably narrow *inside* the API implementation,
rather than being limited to wrapping the entire API call in
try/except.

Cheers,
Nick.

P.S. There are also some use cases where Look-Before-You-Leap is
inherently subject to race conditions, and for those, exceptions are
the only reliable signaling mechanism.

--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia

[Michel Desmoulin]

I doubt very much it will be used for very complexe cases. Just like
comprehensions or ternary expressions, they are a good fit for some
specific use cases, and people will quickly catch on which one.

You rarely see nested comprehensions or ternary expressions while it's
possible to do so, because the Python communality values tend to limit
abuses.

It will be the same for this.

You won't see very complex usages, mostly things like:

val = foo[-1] except IndexError: "bar"
doh = val.attr.other except AttributeError: "default"

Those would already be coded with something similar (or maybe a chain of
getattr()) or next(iter()). It's not a huge risk.

But it's a huge convenience.

[Random832]

On Tue, Sep 13, 2016, at 07:37, Nick Coghlan wrote:
> P.S. There are also some use cases where Look-Before-You-Leap is
> inherently subject to race conditions,

Which use cases *aren't*?

> and for those, exceptions are the only reliable signaling mechanism.

It's entirely possible to design a non-exception interface without race
conditions. dict.get is *almost* a suitable example of such a method.
Remember, at the C level in CPython, throwing an exception *is*
returning a value [typically (PyObject*)NULL for interfaces that
otherwise return an object], along with setting the error indicator. The
only missing piece is an "error value" outside the set of valid values.

Guido's argument here seems to be that exception-based EAFP is not
pythonic.

[Random832]

On Tue, Sep 13, 2016, at 08:44, Michel Desmoulin wrote:
> You won't see very complex usages, mostly things like:
>
> val = foo[-1] except IndexError: "bar"
> doh = val.attr.other except AttributeError: "default"

It occurs to me that if lambda were more lightweight [whether it's the
syntax or the implementation that's the problem is unclear], not only
this but also the ternary operator could have been functions.

val = iexcept(lambda: foo[-1], IndexError, lambda: "bar")

[Chris Angelico]

On Wed, Sep 14, 2016 at 12:14 AM, Random832 <rand...@fastmail.com> wrote:
> On Tue, Sep 13, 2016, at 07:37, Nick Coghlan wrote:
>> P.S. There are also some use cases where Look-Before-You-Leap is
>> inherently subject to race conditions,
>
> Which use cases *aren't*?

Ones in which no external force can affect things. For example:

def frob(spam):
if spam is None:
print("Frobbing nothing")
else:
print("Frobbing some spam: ", spam)
...

You can safely assume that locals won't be changed between the 'is
None' check and the print. I suppose someone could mess around with
thread call stacks, but that'd be seriously insane.

ChrisA

[Rob Cliffe]

On 13/09/2016 12:37, Nick Coghlan wrote:
> On 13 September 2016 at 21:15, Rob Cliffe <rob.c...@btinternet.com> wrote:
>> On 13/09/2016 04:43, Guido van Rossum wrote:
>>> Yeah, that's exactly my point. PEP 463 gives you a shorter way to
>>> catch an exception, so it gives you less motivation to find a way to
>>> write your code (or define your API) that doesn't involve catching
>>> exceptions. But APIs involving exceptions are often inferior to APIs
>>> that don't require exception catching. (Yes, I am aware of __next__()
>>> raising StopIteration -- but that API design usually doesn't require
>>> you to catch it.)
>>>
>> You surprise me. I thought LBYL and EAFP were both approved Python idioms,
>> in some cases one being better, in some cases another, choice to be made on
>> the merits of each case (or the author's preference). I certainly use both
>> (and sometimes time both to see which is faster).
>> Now it sounds as if you're trying to impose a style guide on the world by
>> discouraging the EAFP. And wasn't the discussion general, not about APIs
>> specifically?
> Which is preferable depends greatly on context of use, which is why
> you'll find a lot of Python APIs offer both forms

My point exactly. Random832 echoes my thoughts:

"Guido's argument here seems to be that exception-based EAFP is not
pythonic."

[snip]

> However, blindly catching *all* exceptions from a complex
> subexpression is rarely the right thing to do,

Of course.

> so APIs that only offer
> "this may throw exceptions during normal operation under these
> circumstances" without a convenience wrapper that does the exception
> handling for you can end up being a pain to work with.
>
> PEP 463 makes those APIs less painful to deal with, but at the cost of
> encouraging overly broad exception handlers.

Why? You can catch exactly the same (wide or narrow) range of
exceptions in an exception-catching expression as you can in a
try+except block. Of course
result = (myList[0] except Exception: MyDefault)
is poor code (presumably it should have been written "except
IndexError"), but so is
try:
result = myList[0]
except Exception:
result = MyDefault
ISTM you're giving an exception-catching dog a bad name and hanging
him. Or have I missed something?

And sorry to repeat myself, but we seemed to be having a *general*
discussion about null-coalescing operators, which moved on to PEP 463,
then suddenly Guido is talking about APIs. A lot of the time when I'm
coding, I'm not writing an API, just a program to get a job done. No
doubt, exceptions should be discouraged *in APIs*, but that doesn't make
exceptions, or EAFP, a bad idea per se.

I really don't mean this post to sound hostile, and I'm sorry if it
comes across a bit that way. I'm just surprised at what I'm hearing,
and to be honest, I have a soft spot for PEP 463.
Best wishes
Rob Cliffe

[Chris Angelico]

On Wed, Sep 14, 2016 at 1:27 AM, Rob Cliffe <rob.c...@btinternet.com> wrote:
> And sorry to repeat myself, but we seemed to be having a *general*
> discussion about null-coalescing operators, which moved on to PEP 463, then
> suddenly Guido is talking about APIs. A lot of the time when I'm coding,
> I'm not writing an API, just a program to get a job done. No doubt,
> exceptions should be discouraged *in APIs*, but that doesn't make
> exceptions, or EAFP, a bad idea per se.

PEP 463, by its nature, is talking about the language. Language
features govern and advise API design, which then changes how you
write "a program to get a job done", so it does actually have strong
bearing - just via a couple of levels of indirection.

ChrisA

[MRAB]

On 2016-09-13 16:27, Rob Cliffe wrote:
>
>
> On 13/09/2016 12:37, Nick Coghlan wrote:
>> On 13 September 2016 at 21:15, Rob Cliffe <rob.c...@btinternet.com> wrote:
>>> On 13/09/2016 04:43, Guido van Rossum wrote:
>>>> Yeah, that's exactly my point. PEP 463 gives you a shorter way to
>>>> catch an exception, so it gives you less motivation to find a way to
>>>> write your code (or define your API) that doesn't involve catching
>>>> exceptions. But APIs involving exceptions are often inferior to APIs
>>>> that don't require exception catching. (Yes, I am aware of __next__()
>>>> raising StopIteration -- but that API design usually doesn't require
>>>> you to catch it.)
>>>>
>>> You surprise me. I thought LBYL and EAFP were both approved Python idioms,
>>> in some cases one being better, in some cases another, choice to be made on
>>> the merits of each case (or the author's preference). I certainly use both
>>> (and sometimes time both to see which is faster).
>>> Now it sounds as if you're trying to impose a style guide on the world by
>>> discouraging the EAFP. And wasn't the discussion general, not about APIs
>>> specifically?
>> Which is preferable depends greatly on context of use, which is why
>> you'll find a lot of Python APIs offer both forms
> My point exactly. Random832 echoes my thoughts:
>
> "Guido's argument here seems to be that exception-based EAFP is not
> pythonic."
>
[snip]

I think the point is that exceptions are for, well, exceptional
processing, not normal processing.

Code should be written on the assumption that everything works normally,
e.g. division always returns a result.

_Occasionally_ division will fail, such as attempting to divide by zero,
but that should be uncommon.

If something pretty much always succeeds, use an exception for failure,
but if it sometimes succeeds and sometimes fails, check explicitly for
success or failure.

[Brett Cannon]

This general API discussion is veering off-topic for the subject line. If people would like to continue to discuss it then please start a new thread (if you feel it impacts your view of a null-coalescing operator then please discuss in the new thread and come back to this one).

[Mark E. Haase]

(Replying to multiple posts in this thread)

Guido van Rossum:

Another problem is PEP 505 -- it
is full of discussion but its specification is unreadable due to the
author's idea to defer the actual choice of operators and use a
strange sequence of unicode characters instead.

Hi, I wrote PEP-505. I'm sorry that it's unreadable. The choice of emoji as operators was supposed to be a blatant joke. I'd be happy to submit a new version that is ASCII. Or make any other changes that would facilitate making a decision on the PEP.

As I recall, the thread concluded with Guido writing, "I'll have to think about this," or something to that effect. I had hoped that the next step could be a survey where we could gauge opinions on the various possible spellings. I believe this was how PEP-308 was handled, and that was a very similar proposal to this one.

Most of the discussion on list was really centered around the fact that nobody like the proposed ?? or .? spellings, and nobody could see around that fact to consider whether the feature itself was intrinsically valuable. (This is why the PEP doesn't commit to a syntax.) Also, as unfortunate side effect of a miscommunication, about 95% of the posts on this PEP were written _before_ I submitted a complete draft and so most of the conversation was arguing about a straw man.

David Mertz:

The idea is that we can easily have both "regular" behavior and None coalescing just by wrapping any objects in a utility class... and WITHOUT adding ugly syntax.  I might have missed some corners where we would want behavior wrapped, but those shouldn't be that hard to add in principle.

The biggest problem with a wrapper in practice is that it has to be unwrapped before it can be passed to any other code that doesn't know how to handle it. E.g. if you want to JSON encode an object, you need to unwrap all of the NullCoalesce objects because the json module wouldn't know what to do with them. The process of wrapping and unwrapping makes the resulting code more verbose than any existing syntax.

How much of the time is a branch of the None check a single fallback value or attribute access versus how often a suite of statements within the not-None branch?

I definitely check for None very often also. I'm curious what the breakdown is in code I work with.

There's a script in the PEP-505 repo that can you help you identify code that could be written with the proposed syntax. (It doesn't identify blocks that would not be affected, so this doesn't completely answer your question.)

https://github.com/mehaase/pep-0505/blob/master/find-pep505.py

The PEP also includes the results of running this script over the standard library.

On Sat, Sep 10, 2016 at 1:26 PM, Guido van Rossum <gu...@python.org> wrote:

The way I recall it, we arrived at the perfect syntax (using ?) and
semantics. The issue was purely strong hesitation about whether
sprinkling ? all over your code is too ugly for Python, and in the end
we couldn't get agreement on *that*. Another problem is PEP 505 -- it
is full of discussion but its specification is unreadable due to the
author's idea to defer the actual choice of operators and use a
strange sequence of unicode characters instead.

If someone wants to write a new, *short* PEP that defers to PEP 505
for motivation etc. and just writes up the spec for the syntax and
semantics we'll have a better starting point. IMO the key syntax is
simply one for accessing attributes returning None instead of raising
AttributeError, so that e.g. `foo?.bar?.baz` is roughly equivalent to
`foo.bar.baz if (foo is not None and foo.bar is not None) else None`,
except evaluating foo and foo.bar only once.

On Sat, Sep 10, 2016 at 10:14 AM, Random832 <rand...@fastmail.com> wrote:
> On Sat, Sep 10, 2016, at 12:48, Stephen J. Turnbull wrote:
>> I forget if Guido was very sympathetic to null-coalescing operators,
>> given somebody came up with a good syntax.
>
> As I remember the discussion, I thought he'd more or less conceded on
> the use of ? but there was disagreement on how to implement it that
> never got resolved. Concerns like, you can't have a?.b return None
> because then a?.b() isn't callable, unless you want to use a?.b?() for
> this case, or some people wanted to have "a?" [where a is None] return a
> magic object whose attribute/call/getitem would give no error, but that
> would have to keep returning itself and never actually return None for
> chained operators.

> _______________________________________________
> Python-ideas mailing list
> Python...@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

--
--Guido van Rossum (python.org/~guido)

[Guido van Rossum]

I actually think the spelling is the main stumbling block. The
intrinsic value of the behavior is clear, it's finding an acceptable
spelling that hold back the proposal.

I propose that the next phase of the process should be to pick the
best operator for each sub-proposal. Then we can decide which of the
sub-proposals we actually want in the language, based on a combination
of how important the functionality is and how acceptable we find the
spelling.

--Guido

[Gustavo Carneiro]

For what it's worth, I like the C# syntax with question marks.

It is probably more risky (breaks more code) to introduce a new keyword than a new symbol as operator.

If we have to pick a symbol, it's less confusing if we pick something another language already uses.  There is no shame in copying from other languages.  Many of them copy ideas from Python as well ;-)

Thanks.

--

Gustavo J. A. M. Carneiro

Gambit Research
"The universe is always one step beyond logic." -- Frank Herbert

[Mark E. Haase]

On Fri, Oct 14, 2016 at 12:10 PM, Guido van Rossum <gu...@python.org> wrote:

I propose that the next phase of the process should be to pick the
best operator for each sub-proposal. Then we can decide which of the
sub-proposals we actually want in the language, based on a combination
of how important the functionality is and how acceptable we find the
spelling.

--Guido

I just submitted an updated PEP that removes the emoijs and some other cruft.

How I can help with this next phase? Is a survey a good idea or a bad idea?

[Guido van Rossum]

I'm not usually swayed by surveys -- Python is not a democracy. Maybe
a bunch of longer examples would help all of us see the advantages of
the proposals.

[Nick Coghlan]

On 15 October 2016 at 13:36, Guido van Rossum <gu...@python.org> wrote:
> I'm not usually swayed by surveys -- Python is not a democracy. Maybe
> a bunch of longer examples would help all of us see the advantages of
> the proposals.

Having been previously somewhere between -1 and -0, I've been doing a
lot more data mining and analysis work lately, which has been enough
to shift me to at least +0 and potentially even higher when it comes
to the utility of adding these operators (more on that below).

= Pragmatic aspects =

Regarding the spelling details, my current preferences are as follows:

* None-coalescing operator: x ?or y
* None-severing operator: x ?and y
* None-coalescing augmented assignment: x ?= y
* None-severing attribute access: x?.attr
* None-severing subscript lookup: x?[expr]

(The PEP currently only covers the "or?" and "and?" operator spelling
suggestions, but the latter three suggestions are the same as those in
the current PEP draft)

My rationale for this preference is that it means that "?" is
consistently a pseudo-operator that accepts an expression on the left
and another binary operator (from a carefully restricted subset) on
the right, and the combination is a new short-circuiting binary
operation based on "LHS is not None".

The last three operations can be defined in terms of the first two
(with the usual benefit of avoiding repeated evaluation of the
subexpression):

* None-coalescing augmented assignment: x = x ?or y
* None-severing attribute access: x ?and x.attr
* None-severing subscript lookup: x ?and x[expr]

The first two can then be defined in terms of equivalent if/else
statements containing an "x is not None" clause:

* None-coalescing operator: x if x is not None else y
* None-severing operator: y if x is not None else x

Importantly, the normal logical and/or can be expanded in terms of
if/else in exactly the same way, only using "bool(x)" instead of "x is
not None":

* Logical or: x if x else y
* Logical and: y if x else x

= Language design philosophy aspects =

Something I think is missing from the current PEP is a high level
explanation of the *developer problem* that these operators solve -
while the current PEP points to other languages as precedent, that
just prompts the follow on question "Well, why did *they* add them,
and does their rationale also apply to Python?". Even the current
motivating examples don't really cover this, as they're quite tactical
in nature ("Here is how this particular code is improved by the
proposed change"), rather than explaining the high level user benefit
("What has changed in the surrounding technology environment that
makes us think this is a user experience design problem worth changing
the language definition to help address *now* even though Python has
lived happily without these operators for 25+ years?")

With conditional expressions, we had the clear driver that folks were
insisting on using (and teaching!) the "and/or" hack as a workaround,
and introducing bugs into their code as a result, whereas we don't
have anything that clear-cut for this proposal (using "or" for
None-coalescing doesn't seem to be anywhere near as popular as
"and/or" used to be as an if/else equivalent).

My point of view on that is that one of the biggest computing trends
in recent years is the rise of "semi-structured data", where you're
passing data around in either JSON-compatible data structures, or
comparable structures mapped to instances and attributes, and all
signs point to that being a permanent state change in the world of
programming rather than merely being a passing fad. The world itself
is fuzzy and ambiguous, and learning to work effectively with
semi-structured data better reflects that ambiguity rather than
forcing a false precision for the sake of code simplification. When
you're working in that kind of context, encountering "None" is
typically a shorthand for "This entire data subtree is missing, so
don't try to do anything with it", but if it *isn't* None, you can
safely assume that all the mandatory parts of that data segment will
be present (no matter how deeply nested they are).

To help explain that, it would be useful to mention not only the
corresponding operators in other languages, but also the changes in
data storage practices, like PostgreSQL's native support for JSON
document storage and querying (
https://www.postgresql.org/docs/9.4/static/functions-json.html ) as
well as the emergence/resurgence of hierarchical document storage
techniques and new algorithms for working with them.

However, it's also the case that where we *do* have a well understood
and nicely constrained problem, it's still better to complain loudly
when data is unexpectedly missing, rather than subjecting ourselves to
the pain of having to deal with detecting problems with our data far
away from where we introduced those problems. A *lot* of software
still falls into that category, especially custom software written to
meet the needs of one particular organisation.

My current assumption is that those of us that now regularly need to
deal with semi-structured data are thinking "Yes, these additions are
obviously beneficial and improve Python's expressiveness, if we can
find an acceptable spelling". Meanwhile, folks dealing primarily with
entirely structured or entirely unstructured data are scratching their
heads and asking "What's the big deal? How could it ever be worth
introducing more line noise into the language just to make this kind
of code easier to write?"

Even the PEP's title is arguably a problem on that front - "None-aware
operators" is a proposed *solution* to the problem of making
semi-structured data easier to work with in Python, and hence reads
like a solution searching for a problem to folks that don't regularly
encounter these issues themselves.

Framing the problem that way also provides a hint on how we could
*document* these operations in the language reference in a readily
comprehensible way: "Operators for working with semi-structured data"

Regards,
Nick.

--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia

[Ivan Levkivskyi]

15 Жов 2016 08:11 "Nick Coghlan" <ncog...@gmail.com> пише:

>
> On 15 October 2016 at 13:36, Guido van Rossum <gu...@python.org> wrote:
> > I'm not usually swayed by surveys -- Python is not a democracy. Maybe
> > a bunch of longer examples would help all of us see the advantages of
> > the proposals.
>
> Having been previously somewhere between -1 and -0, I've been doing a
> lot more data mining and analysis work lately, which has been enough
> to shift me to at least +0 and potentially even higher when it comes
> to the utility of adding these operators (more on that below).
>

It is a real pleasure to read Nick's posts, and here he says _exactly_ what I wanted to say, but in a much clearer way than I could.

(Disclaimer: I am working with semi-structured data most of time)

--
Ivan

[Sven R. Kunze]

On 15.10.2016 08:10, Nick Coghlan wrote:
> However, it's also the case that where we *do* have a well understood
> and nicely constrained problem, it's still better to complain loudly
> when data is unexpectedly missing, rather than subjecting ourselves to
> the pain of having to deal with detecting problems with our data far
> away from where we introduced those problems. A *lot* of software
> still falls into that category, especially custom software written to
> meet the needs of one particular organisation.

Definitely true. Stricter rules are similar to "fail early", "no errors
should pass silently" and the like. This stance is conveyed by Python as
long as I know it.

> My current assumption is that those of us that now regularly need to
> deal with semi-structured data are thinking "Yes, these additions are
> obviously beneficial and improve Python's expressiveness, if we can
> find an acceptable spelling". Meanwhile, folks dealing primarily with
> entirely structured or entirely unstructured data are scratching their
> heads and asking "What's the big deal? How could it ever be worth
> introducing more line noise into the language just to make this kind
> of code easier to write?"

That's where I like to see a common middle ground between those two
sides of the table.


I need to work with both sides for years now. In my experience, it's
best to avoid semi-structured data at all to keep the code simple. As we
all know and as you described, the world isn't perfect and I can only
agree. However, what served us best in recent years, is to keep the
"semi-" out of the inner workings of our codebase. So, handling "semi-"
at the system boundary proved to be a reliable way of not breaking
everything and of keeping our devs sane.

I am unsure how to implement such solution, whether via PEP8 or via the
proposal's PEP. It somehow reminds me of the sans-IO idea where the core
logic should be simple/linear code and the difficult/problematic issues
are solved at the systems boundary.


This said, let me put it differently by using an example. I can find
None-aware operators very useful at the outermost function/methods of a
process/library/class/module:

class FanzyTool:
def __init__(self, option1=None, option2=None, ...):
# what happens when option6 and option7 are None
# and it only matters when option 3 is not None
# but when ...

Internal function/methods/modules/classes and even processes/threads
should have a clear, non-wishy-washy way of input and output (last but
not least also to do unit-testing on relatively sane level).

def _append_x(self, s):
return s + 'x' # strawman operation

Imagine, that s is something important to be passed around many times
inside of "FanzyTool". The whole process usually makes no sense at all,
when s is None. And having each internal method checking for None is
getting messy fast.


I hope we can also convey this issue properly when we find an
appropriate syntax.

> Even the PEP's title is arguably a problem on that front - "None-aware
> operators" is a proposed *solution* to the problem of making
> semi-structured data easier to work with in Python, and hence reads
> like a solution searching for a problem to folks that don't regularly
> encounter these issues themselves.
>
> Framing the problem that way also provides a hint on how we could
> *document* these operations in the language reference in a readily
> comprehensible way: "Operators for working with semi-structured data"

That's indeed an extraordinarily good title as it describes best what we
intend it to be used for (common usage scenarios). +1

Regards,
Sven

[Barry Warsaw]

On Oct 15, 2016, at 04:10 PM, Nick Coghlan wrote:

>Having been previously somewhere between -1 and -0, I've been doing a
>lot more data mining and analysis work lately, which has been enough
>to shift me to at least +0 and potentially even higher when it comes
>to the utility of adding these operators (more on that below).

I'm sympathetic to (some of) the goals of PEP 505, as these issues do
occasionally annoy me. But I'm not entirely convinced they are common enough
or annoying enough to warrant special syntax, and I *really* dislike the
introduction of a ? operator for these purposes. I'm also concerned about
adopting too much generality muddling up what I think should be a narrowly
targeted improvement to readability.

The other thing to note is that, while I often use ternary operators for this
now, checking against None isn't always the sole conditional. E.g.

self.chain = (chain
if chain is None or IChain.providedBy(chain)
else config.chains[chain])

That being said, null-aware member access (NAMA) would be pretty handy
occasionally. I'm less sure about the other forms. For me, the biggest
benefit of NAMA is the short-circuiting of chained attribute access.

I don't like the operator syntax because I find it less readable (harder for
the eye to pick out), and because it isn't a completely obvious operation.
But also because I generally want to chase the attributes all-or-nothing. For
example, foo.bar.baz.qux but only if all the intermediary attributes resolve
to non-Nones. I don't want to have to write foo.?bar.?baz.?qux

I tried playing around with new keywords such as 'when' and 'unless', which
seem a little nice although not a perfect fit.

thing = foo.bar.baz.qux unless None

thing = unless None then foo.bar.baz.qux

thing = when foo.bar.baz.qux

thing = foo.bar.baz.qux when not None

I do like the idea of a keyword more than an operator, and disagree that a new
keyword can't be introduced until Python 4. That's why we have __future__!

Anyway, that's my $0.02. I trust Guido to DTPT (do the Pythonic thing :),
even if that means rejecting the PEP.

Cheers,
-Barry

[Nick Badger]

Am Freitag, 14. Oktober 2016 23:11:48 UTC-7 schrieb Nick Coghlan:

Regarding the spelling details, my current preferences are as follows:

* None-coalescing operator: x ?or y
* None-severing operator: x ?and y
* None-coalescing augmented assignment: x ?= y
* None-severing attribute access: x?.attr
* None-severing subscript lookup: x?[expr]
 

This is, more or less, the syntax added in Nick's PEP 531 draft. The reddit discussion about it raised some pretty major concerns about clarity, and I have to admit, I think if you're learning Python as a first language, the ?and, ?else, x?.attr, etc syntax is likely to be very confusing. For me personally, combining a new operator "?" with existing keywords like "and" or "else" just does not make any intuitive sense. I definitely see the value, though, in particular of None-severing, especially as a tool to explicitly specify which attr can be missing -- ie, disambiguating which attribute is missing in a foo.bar.baz lookup (the alternative to which is nested try: except AttributeError: blocks, which gets very messy very quickly). I'm on board with the idea, and I can absolutely imagine using it in my code, but I disagree on the spelling.

A thought I had (perhaps more readable in a reddit comment) is to condense everything into a single "?" symbol, used for:

+ Coalescing binary operator: foo ? bar
+ Coalescing augmented assignment operator: foo ?= bar
+ Severing unary operator: ?foo

Pseudocode binary operator examples:

>>> foo_exists ? bar_never_evaluated
foo_exists

>>> foo_missing ? foo_exists
foo_exists

>>> foo_missing ? bar_missing
foo_missing

Pseudocode augmented examples:

>>> foo_exists = 'foo'
>>> foo_exists ?= bar_never_evaluated
>>> foo_exists == 'foo'
True

>>> foo = Missing
>>> bar_exists = 'bar'
>>> foo ?= bar_exists
>>> foo == 'bar'
True

>>> foo = None
>>> bar_missing = Missing
>>> foo ?= bar_missing
>>> foo == None
True

Pseudocode unary examples:

>>> ?(foo_exists).bar.baz
foo_exists.bar.baz
>>> ?(foo_exists)[bar][baz]
foo_exists[bar][baz]

>>> ?(foo_missing).bar.baz
Missing
>>> ?(foo_missing)[bar][baz]
Missing

>>> ?(foo_exists).bar.baz_missing
Traceback...
AttributeError: <foo_exists.bar> object has no attribute 'baz_missing'
>>> ?(foo_exists)[bar][baz_missing]
Traceback...
KeyError: 'baz_missing'

>>> ?(foo_missing).bar.baz_missing
Missing
>>> ?(foo_missing)[bar][baz_missing]
Missing

I personally think that's substantially more readable, but I suppose that's at least somewhat a matter of personal preference.

[Joonas Liik]

perhaps just having a utility function can get us some of the way there..

#may error
r = a.b.x.z

# will default to None
r = a?.b?.x?.z
r = get_null_aware(a, "b.x.z") # long but no new syntax, can be
implemented today.

[Random832]

On Thu, Oct 27, 2016, at 11:27, Joonas Liik wrote:
> perhaps just having a utility function can get us some of the way there..
>
> #may error
> r = a.b.x.z
>
> # will default to None
> r = a?.b?.x?.z

If a.b can't or shouldn't be None, this should be a?.b.x.z

I'm not certain how your utility function is supposed to differentiate
this case, or handle subscripts or method calls.

[Barry Warsaw]

On Oct 27, 2016, at 06:27 PM, Joonas Liik wrote:

>perhaps just having a utility function can get us some of the way there..
>
>#may error
>r = a.b.x.z
>
># will default to None
>r = a?.b?.x?.z
>r = get_null_aware(a, "b.x.z") # long but no new syntax, can be
>implemented today.

You could probably do this by extending operator.attrgetter() to take an
optional 'coalesce' keyword. It wouldn't be super pretty, but it has the
advantage of no magical new syntax. E.g. your example would be:

from operator import attrgetter
r = attrgetter('b.x.z', coalesce=True)

That might be good enough for honestly how rare I think this use case is.
(Similarly with itemgetter().)

Cheers,
-Barry

[Nick Badger]

The problem with doing that is that it's ambiguous. There's no way of telling which attribute is allowed to coalesce.

I think one of the best arguments for a coalescing operator in Python is that it allows you to be more explicit, without the hassle of nested try: except AttributeError blocks. You lose that with something like attrgetter('b.x.z', coalesce=True) -- it would behave identically, regardless of whether b, x, or z were missing, which is (oftentimes) not what you want.

Nick Badger

https://www.muterra.io

https://www.nickbadger.com

[Barry Warsaw]

On Oct 27, 2016, at 07:37 PM, Nick Badger wrote:

>The problem with doing that is that it's ambiguous. There's no way of
>telling which attribute is allowed to coalesce.

You could of course support exactly the same syntax being proposed as a
language change, e.g.

from operator import attrgetter
r = attrgetter('b?.x?.z')

and then you wouldn't even need the `coalesce` argument.

Cheers,
-Barry

[Eric Snow]

On Fri, Oct 28, 2016 at 7:13 AM, Barry Warsaw <ba...@python.org> wrote:
> You could of course support exactly the same syntax being proposed as a
> language change, e.g.
>
> from operator import attrgetter
> r = attrgetter('b?.x?.z')
>
> and then you wouldn't even need the `coalesce` argument.

+1

-eric

[Gustavo Carneiro]

The main drawback of this type of approach is that code checking tools will hardly ever support checking expressions inside the string like that.  Also, you don't get proper syntax highlighting, code completion, etc.

You can do anything you want by writing another programming language that is passed as string to a function, but that is not the same thing as having a proper syntax, is it?  Just like type annotations with mypy: sure, you can add type annotations in comments, but it's not the same...

[Barry Warsaw]

On Oct 28, 2016, at 03:24 PM, Gustavo Carneiro wrote:

>The main drawback of this type of approach is that code checking tools will
>hardly ever support checking expressions inside the string like that.
>Also, you don't get proper syntax highlighting, code completion, etc.
>
>You can do anything you want by writing another programming language that
>is passed as string to a function, but that is not the same thing as having
>a proper syntax, is it? Just like type annotations with mypy: sure, you
>can add type annotations in comments, but it's not the same...

The bar for adding new language syntax is, and must be, high. Every new bit
of syntax has a cost, so it has to be worth it. Guido deemed type annotations
to be worth it and he may do the same for null coalescing operators. I don't
personally think the need is so great or the use cases so common to incur that
cost, but I'm just one opinion.

The advantage of lower-cost approaches such as adopting the syntax in
attrgetter() is that you piggyback on an existing API. Then you can use that
as an experiment to see whether you really do solve enough problems in Python
for a syntax change to be worth it. It's a lot like the ability to create
properties and such before the syntactic sugar of decorators was added. I
think that feature's pre-syntax popular and utility proved that the cost of
adding syntax was worth it.

Cheers,
-Barry

[Mark E. Haase]

On Fri, Oct 14, 2016 at 11:36 PM, Guido van Rossum <gu...@python.org> wrote:

I'm not usually swayed by surveys -- Python is not a democracy. Maybe
a bunch of longer examples would help all of us see the advantages of
the proposals.

I understand. You said the next phase should be to pick the best operator for each sub-proposal but I'm not sure how I can help with that. If there's something I can do, let me know and I'm happy to try to do it.

In terms of "bunch of longer examples", what did you have in mind? I could take some popular library and rewrite a section of it with the proposed operators, but that would depend on the response to the previous paragraph.

[Stephen J. Turnbull]

Mark E. Haase writes:

> In terms of "bunch of longer examples", what did you have in mind?
> I could take some popular library and rewrite a section of it with
> the proposed operators, but that would depend on the response to
> the previous paragraph.

I gather you think you have a deadlock here. The way to break it is
to just do it. Pick a syntax and do the rewriting. My memory of some
past instances is that many of the senior devs (especially Guido) will
"see through the syntax" to evaluate the benefits of the proposal,
even if they've said they don't particularly like the initially-
proposed syntax. Unfortunately here the most plausible syntax is one
that Guido has said he definitely doesn't like: using '?'. The
alternatives are pretty horrible (a Haskell-like 'maybe' keyword, or
the OPEN SQUARE character used by some logicians in modal logic -- the
problem with the latter is that for many people it may not display at
all with their font configurations, or it may turn into mojibake in
email.

OTOH, that case was an astral character -- after Guido announced his
opposition to '?', the poster used PILE OF POO as the operator. OPEN
SQUARE is in the basic multilingual plane, so probably is OK if the
recipient can handle Unicode. '?' vs. '□': maybe that helps narrow
the choice set?

[Steven D'Aprano]

On Sat, Oct 29, 2016 at 11:02:36AM +0900, Stephen J. Turnbull wrote:

> Unfortunately here the most plausible syntax is one
> that Guido has said he definitely doesn't like: using '?'. The
> alternatives are pretty horrible (a Haskell-like 'maybe' keyword, or
> the OPEN SQUARE character used by some logicians in modal logic -- the
> problem with the latter is that for many people it may not display at
> all with their font configurations, or it may turn into mojibake in
> email.

I think you mean WHITE SQUARE? At least, I can not see any "OPEN SQUARE"
code point in Unicode, and the character you use below □ is called WHITE
SQUARE.

> OTOH, that case was an astral character -- after Guido announced his
> opposition to '?', the poster used PILE OF POO as the operator. OPEN
> SQUARE is in the basic multilingual plane, so probably is OK if the
> recipient can handle Unicode. '?' vs. '□': maybe that helps narrow
> the choice set?

I cannot wait for the day that we can use non-ASCII operators. But I
don't think that day has come: it is still too hard for many people
(including me) to generate non-ASCII characters at the keyboard, and
font support for some of the more useful ones are still inconsistent or
lacking.

For example, we don't have a good literal for empty sets. How about ∅?
Sadly, in my mail client and in the Python REPR, it displays as a
"missing glyph" open rectangle. And how would you type it?

Ironically, WHITE SQUARE does display, but it took me a while to realise
because at first I thought it too was the missing glyph character. And I
still have no idea how to type it.

Java, I believe, allows you to enter escape sequences in source code,
not just in strings. So we could hypothetically allow one of:

myobject\N{WHITE SQUARE}attribute
myobject\u25a1attribute

as a pure-ASCII way of getting

myobject□attribute

but really, who is going to do that? It is bad enough when strings
contain escape sequences, but source code?

So even though I *want* to use non-ASCI operators, I have to admit that
I *can't* realistically use non-ASCII operators. Not yet.

Wishing-that-somebody-can-prove-me-wrong-ly y'rs,

--
Steve

[Paul Moore]

On 29 October 2016 at 07:30, Steven D'Aprano <st...@pearwood.info> wrote:
> So even though I *want* to use non-ASCI operators, I have to admit that
> I *can't* realistically use non-ASCII operators. Not yet.

Personally, I'm not even sure I want non-ASCII operators until
non-ASCII characters are common, and used without effort, in natural
language media such as email (on lists like this), source code
comments, documentation, etc.

For better or worse, it may be emoji that drive that change ;-)

Paul

[Stephen J. Turnbull]

Steven d'Aprano writes:

> I think you mean WHITE SQUARE? At least, I can not see any "OPEN
> SQUARE" code point in Unicode, and the character you use below □
> is called WHITE SQUARE.

You're right, I just used a common Japanese name for it. I even
checked the table to make sure it was BMP but didn't notice the proper
name which is written right there. Sorry for the confusion.

Paul Moore writes:

> Personally, I'm not even sure I want non-ASCII operators until
> non-ASCII characters are common, and used without effort, in natural
> language media such as email (on lists like this), source code
> comments, documentation, etc.

The 3 billion computer users (and their ancestors) who don't live in
the U.S. or Western Europe have been using non-ASCII, commonly,
without effort, in natural language media on lists like this one for
up to 5 decades now. In my own experience, XEmacs lists have
explictly allowed Japanese and Russian since 1998, and used to see the
occasional posts in German, French and Spanish, with no complaints of
mojibake or objections that I can recall. And I have maintained
XEmacs code containing Japanese identifiers, both variables and
functions, since 1997.

I understand why folks are reluctant, but face it, the technical
issues were solved before half our users were born. It's purely a
social problem now, and pretty much restricted to the U.S. at that.

> For better or worse, it may be emoji that drive that change ;-)

I suspect that the 100 million or so Chinese, Japanese, Korean, and
Indian programmers who have had systems that have no trouble
whatsoever handling non-ASCII for as long they've used computers will
drive that change.

[Paul Moore]

On 29 October 2016 at 18:19, Stephen J. Turnbull

<turnbull....@u.tsukuba.ac.jp> wrote:
>> For better or worse, it may be emoji that drive that change ;-)
>
> I suspect that the 100 million or so Chinese, Japanese, Korean, and
> Indian programmers who have had systems that have no trouble
> whatsoever handling non-ASCII for as long they've used computers will
> drive that change.

My apologies. You are of course absolutely right.

I'm curious to know how easy it is for Chinese, Japanese, Korean and
Indian programmers to use *ASCII* characters. I have no idea in
practice whether the current basically entirely-ASCII nature of
programming languages is as much a problem for them as I imagine
Unicode characters would be for me. I really hope it isn't...

Paul

[Mikhail V]

On 29 October 2016 at 18:19, Stephen J. Turnbull

<turnbull.stephen.fw at u.tsukuba.ac.jp> wrote:
>> For better or worse, it may be emoji that drive that change ;-)
>>
>> I suspect that the 100 million or so Chinese, Japanese, Korean, and
>> Indian programmers who have had systems that have no trouble
>> whatsoever handling non-ASCII for as long they've used computers will
>> drive that change.

>My apologies. You are of course absolutely right.
>
>I'm curious to know how easy it is for Chinese, Japanese, Korean and
>Indian programmers to use *ASCII* characters. I have no idea in
>practice whether the current basically entirely-ASCII nature of
>programming languages is as much a problem for them as I imagine
>Unicode characters would be for me. I really hope it isn't...
>
>Paul

The only way to do it
http://ic.pics.livejournal.com/ibigdan/8161099/4947638/4947638_original.jpg

Seriously, as a russian, I never had any problems with
understanding that I should not go that far.

I don't know of any axamples when using translit
caused any personal problems in online conversations,
unless it comes to quarrels and one tries to insult
others for using translit.
But russians are generally more minimalistically
tuned than many other folks.

As for returning non null, I suppose most readable way
would be something like:

non_null(a,b,c...)

(sorry if I am missing the whole discussion topic, can easily happen
with me since it is really mind blowing, why I would ever need it)

Mikhail

[Stephen J. Turnbull]

Paul Moore writes:
> On 29 October 2016 at 18:19, Stephen J. Turnbull
> <turnbull....@u.tsukuba.ac.jp> wrote:
> >> For better or worse, it may be emoji that drive that change ;-)
> >
> > I suspect that the 100 million or so Chinese, Japanese, Korean, and
> > Indian programmers who have had systems that have no trouble
> > whatsoever handling non-ASCII for as long they've used computers will
> > drive that change.
>
> My apologies. You are of course absolutely right.

tl;dr: A quick apology for the snark, and an attempt at FUD reduction.
Using non-ASCII characters will involve some cost, but there are real
benefits, and the fear and loathing often evoked by the prospect is
unnecessary. I'm not ready to advocate introduction *right* now, but
"never" isn't acceptable either. :-)

On with the show:

"Absolutely" is more than I deserve, as I was being a bit snarky.

That said, Ed Yourdon wrote a book in 1990 or so with the
self-promoting title of "Decline and Fall of the American
Programmer"[1] in which he argued that for many kinds of software
outsourcing to China, India, or Ireland got you faster, better,
cheaper, and internationalized, with no tradeoffs. (The "and
internationalized" is my hobby horse, it wasn't part of Yourdon's
thesis.) He later recanted the extremist doomsaying, but a quick
review of the fraction of H1B visas granted to Asian-origin
programmers should convince you that USA/EUR/ANZ doesn't have a
monopoly of good-to-great programming (probably never did, but that's
a topic for a different thread). Also note that in Japan, without
controlling for other factors, just the programming language used most
frequently, Python programmers are the highest paid among developers
in all languages with more than 1% of the sample (and yes, that
includes COBOL!) To the extent that internationalization matters to a
particular kind of programming, these programmers are better placed
for those jobs, I think. And while in many cases "on site" has a big
advantage (so you can't telecommute from Bangalore, you need that H1B
which is available in rather restrictive number), more and more
outsourcing does cross oceans so potential competition is immense.

There is a benefit to increasing our internationalization in backward-
incompatible ways. And that benefit is increasing both in magnitude
and in the number of Python developers who will receive it.

> I'm curious to know how easy it is for Chinese, Japanese, Korean and
> Indian programmers to use *ASCII* characters. I have no idea in
> practice whether the current basically entirely-ASCII nature of
> programming languages is as much a problem for them

Characters are zero problem for them. The East Asian national
standards all include the ASCII repertoire, and some device (usually
based on ISO 2022 coding extensions rather than UTF-8) for allowing
ASCII to be one-byte, even if the "local" characters require two or
more bytes. I forget if India's original national standard also
included an ASCII subset, but they switched over to Unicode quite
early[2], so UTF-8 does the trick for them. English (the language) is a
much bigger issue.

Most Indians, of course, have little trouble with the derived-from-
English nature of much programming syntax and library identifiers, and
the Asians all get enough training in both (very) basic English and
rote memorization that handling English-derived syntax and library
nomenclature is not a problem.

However, reading and especially creating documentation can be
expensive and inaccurate. At least in Japanese, "straightforward"
translations are often poor, as nuances are lost. E.g., a literal
Japanese translation from English requires many words to indicate the
differences a simple "a" vs. "the" vs. "some" indicates in English.
Mostly such nuances can be expressed economically by restructuring a
whole paragraph, but translators rarely bother and often seem unaware
of the issues. Many Japanese programmers' use of articles is literally
chaotic: it's deterministic but appears random to all but the most
careful analysis.[3]

> as I imagine Unicode characters would be for me. I really hope it
> isn't...

I think your imagination is running away with you. While I understand
how costly it is for those over the age of 12 to develop new habits
(I'm 58, and painfully aware of how frequently I balk at learning
anything new no matter how productivity-enhancing it is likely to be,
and how much more slowly it becomes part of my repertoire), the number
of new things you would need to learn would be few, and frequently
enough used, at least in Python. It's hard enough to get Guido (and
the other Masters of Pythonic Language Design) to sign on to new ASCII
syntax; even if in principle non-ASCII were to be admitted, I suspect
the barrier there would be even higher.

Most of Unicode is irrelevant to everybody. Mathematicians use only a
small fraction of the math notation available to them -- it's just
that it's a different small fraction for each field. The East Asians
need a big chunk (I would guess that educated Chinese and Japanese
encounter about 10,000 characters in "daily life" over a lifetime,
while those encountered at least once a week number about 3000), but
those that need to be memorized are a small minority (less than 5%) of
the already defined Unicode repertoire.

For Western programmers, the mechanics are almost certainly there.
Every personal computer should have at least one font containing all
characters defined in the Basic Multilingual Plane, and most will have
chunks of the astral planes (emoji, rare math symbols, country flags,
...). Even the Happy Hacker keyboard has enough mode keys (shift,
control, ...) to allow defining "3-finger salutes" for commonly-used
characters not on the keycaps -- in daily life if you don't need a
input method now, you won't need one if Python decides to use WHITE
SQUARE to represent an operation you frequently use -- just an extra
"control key combo" like the editing control keys (eg, for copy, cut,
paste, undo) that aren't marked on any keyboard I have.

I'm *not* advocating *imposing* the necessary effort on anyone right
now. I just want to reduce the FUD associated with the prospect that
it *might* be imposed on *you*, so that you can evaluate the benefits
in light of the real costs. They're not zero, but they're unlikely to
ruin your whole day, every day, for months.[4]

"Although sometimes never is better than *right* now" doesn't apply
here. :-)


Footnotes:
[1] India is a multiscript country, so faces the same pressure for a
single, internationally accepted character set as the whole world
does, albeit at a lower level.

[2] "The American Programmer" was the name of Yourdon's consultancy's
newsletter to managers of software projects and software development
organizations.

[3] Of course the opposite is true when I write Japanese. In
particular, there's a syntactic component called "particle" (the
closest English equivalent is "preposition", but particles have much
more general roles) that I'm sure my usage is equally chaotic from the
point of view of a native speaker of Japanese -- even after working in
the language for 25 years! N.B. I'm good enough at the language to
have written grant proposals that were accepted in it -- and still my
usage of particles is unreliable.

[4] Well, if your role involves teaching other programmers, their
pushback could be a long-lasting irritant. :-(

[Paul Moore]

On 30 October 2016 at 07:00, Stephen J. Turnbull

My point wasn't so much about dealing with the character set of
Unicode, as it was about physical entry of non-native text. For
example, on my (UK) keyboard, all of the printed keycaps are basically
used. And yet, I can't even enter accented letters from latin-1 with a
standard keypress, much less extended Unicode. Of course it's possible
to get those characters (either by specialised mappings in an editor,
or by using an application like Character Map) but there's nothing
guaranteed to work across all applications. That's a hardware and OS
limitation - the hardware only has so many keys to use, and the OS
(Windows, in my case) doesn't support global key mapping (at least not
to my knowledge, in a user-friendly manner - I'm excluding writing my
own keyboard driver :-)) My interest in East Asian experience is at
least in part because the "normal" character sets, as I understand it,
are big enough that it's impractical for a keyboard to include a
plausible basic range of characters, so I'm curious as to what the
physical process is for typing from a vocabulary of thousands of
characters on a sanely-sized keyboard.

In mentioning emoji, my main point was that "average computer users"
are more and more likely to want to use emoji in general applications
(emails, web applications, even documents) - and if a sufficiently
general solution for that problem is found, it may provide a solution
for the general character-entry case. (Also, I couldn't resist the
irony of using a :-) smiley while referring to emoji...) But it may be
that app-specific solutions (e.g., the smiley menu in Skype) are
sufficient for that use case. Or the typical emoji user is likely to
be using a tablet/phone rather than a keyboard, and mobile OSes have
included an emoji menu in their on-screen keyboards.

Coming back to a more mundane example, if I need to type a character
like é in an email, I currently need to reach for Character Map and
cut and paste it. The same is true if I have to type it into the
console. That's a sufficiently annoying stumbling block that I'm
inclined to avoid it - using clumsy workarounds like referring to "the
OP" rather than using their name. I'd be fairly concerned about
introducing non-ASCII syntax into Python while such stumbling blocks
remain - the amount of code typed outside of an editor (interactive
prompt, emails, web applications like Jupyter) mean that editor-based
workarounds like custom mappings are only a partial solution.

But maybe you are right, and it's just my age showing. The fate of APL
probably isn't that relevant these days :-) (or ☺ if you prefer...)

Paul

[Chris Angelico]

On Sun, Oct 30, 2016 at 11:22 PM, Paul Moore <p.f....@gmail.com> wrote:
> In mentioning emoji, my main point was that "average computer users"
> are more and more likely to want to use emoji in general applications
> (emails, web applications, even documents) - and if a sufficiently
> general solution for that problem is found, it may provide a solution
> for the general character-entry case.

Before Unicode emoji were prevalent, ASCII emoticons dominated, and
it's not uncommon for multi-character sequences to be automatically
transformed into their corresponding emoji. It isn't hard to set
something up that does these kinds of transformations for other
Unicode characters - use trigraphs for clarity, and type "/:0" to
produce "∅". Or whatever's comfortable for you. Maybe rig it on
Ctrl-Alt-0, if you prefer shift-key sequences.

ChrisA

[Paul Moore]

On 30 October 2016 at 12:31, Chris Angelico <ros...@gmail.com> wrote:
> On Sun, Oct 30, 2016 at 11:22 PM, Paul Moore <p.f....@gmail.com> wrote:
>> In mentioning emoji, my main point was that "average computer users"
>> are more and more likely to want to use emoji in general applications
>> (emails, web applications, even documents) - and if a sufficiently
>> general solution for that problem is found, it may provide a solution
>> for the general character-entry case.
>
> Before Unicode emoji were prevalent, ASCII emoticons dominated, and
> it's not uncommon for multi-character sequences to be automatically
> transformed into their corresponding emoji. It isn't hard to set
> something up that does these kinds of transformations for other
> Unicode characters - use trigraphs for clarity, and type "/:0" to
> produce "∅". Or whatever's comfortable for you. Maybe rig it on
> Ctrl-Alt-0, if you prefer shift-key sequences.

It's certainly not difficult, in principle. I have (had, I lost it in
an upgrade recently...) a little AutoHotkey program that interpreted
Vim-style digraphs in any application that needed them. But my point
was that we don't want to require people to write such custom
utilities, just to be able to write Python code. Or is the feeling
that it's acceptable to require that?

Paul

[Nick Coghlan]

On 30 October 2016 at 23:39, Paul Moore <p.f....@gmail.com> wrote:
> It's certainly not difficult, in principle. I have (had, I lost it in
> an upgrade recently...) a little AutoHotkey program that interpreted
> Vim-style digraphs in any application that needed them. But my point
> was that we don't want to require people to write such custom
> utilities, just to be able to write Python code. Or is the feeling
> that it's acceptable to require that?

Getting folks used to the idea that they need to use the correct kinds
of quotes is already challenging :)

However, the main issue is the one I mentioned in PEP 531 regarding
the "THERE EXISTS" symbol: Python and other programming languages
re-use "+", "-", "=" etc because a lot of folks are already familiar
with them from learning basic arithmetic. Other symbols are used in
Python because they were inherited from C, or were relatively
straightforward puns on such previously inherited symbols.

What this means is that there aren't likely to be many practical gains
in using the "right" symbol for something, even when it's already
defined in Unicode, as we expect the number of people learning that
symbology *before* learning Python to be dramatically smaller than the
proportion learning Python first and the formal mathematical symbols
later (if they learn them at all).

This means that instead of placing more stringent requirements on
editing environments for Python source code in order to use non-ASCII
input symbols, we're still far more likely to look to define a
suitable keyword, or assign a relatively arbitrary meaning to an ASCII
punctuation symbol (and that's assuming we accept that a proposal will
see sufficient use to be worthy of new syntax in the first place,
which is far from being a given).

Cheers,

Nick.

--
Nick Coghlan | ncog...@gmail.com | Brisbane, Australia