Null coalescing operators

[Mark Haase]

StackOverflow has many questions on the topic of null coalescing operators in Python, but I can't find any discussions of them on this list or in any of the PEPs. Has the addition of null coalescing operators into Python ever been discussed publicly?

Python has an "or" operator that can be used to coalesce false-y values, but it does not have an operator to coalesce "None" exclusively.

C# has nice operators for handling null: "??" (null coalesce), "?." (null-aware member access), and "?[]" (null-aware index access). They are concise and easy to reason about. I think these would be a great addition to Python.

As a motivating example: when writing web services, I often want to change the representation of a non-None value but also need to handle None gracefully. I write code like this frequently: 

    response = json.dumps({

        'created': created.isoformat() if created is not None else None,

        'updated': updated.isoformat() if updated is not None else None,

        ...

    })

With a null-aware member access operator, I could write this instead:

    response = json.dumps({

        'created': created?.isoformat(),

        'updated': updated?.isoformat(),

        ...

    })

I can implement this behavior myself in pure Python, but it would be (a) nice to have it the in the standard library, and (b) even nicer to have an operator in the language, since terseness is the goal.

I assume that this has never been brought up in the past because it's so heinously un-Pythonic that you'd have to be a fool to risk the public mockery and shunning associated with asking this question. Well, I guess I'm that fool: flame away...

Thanks,

Mark

[Trent Nelson]

On Fri, Sep 18, 2015 at 10:42:59AM -0700, Mark Haase wrote:
> StackOverflow has many questions
> <http://stackoverflow.com/search?q=%5Bpython%5D+null+coalesce> on the

> topic of null coalescing operators in Python, but I can't find any
> discussions of them on this list or in any of the PEPs. Has the
> addition of null coalescing operators into Python ever been discussed
> publicly?
>
> Python has an "or" operator that can be used to coalesce false-y
> values, but it does not have an operator to coalesce "None"
> exclusively.

Hmmm, I use this NullObject class when I want to do stuff similar to what
you've described:

class NullObject(object):
"""
This is a helper class that does its best to pretend to be forgivingly
null-like.
>>> n = NullObject()
>>> n
None
>>> n.foo
None
>>> n.foo.bar.moo
None
>>> n.foo().bar.moo(True).cat().hello(False, abc=123)
None
>>> n.hornet(afterburner=True).shotdown(by=n().tomcat)
None
>>> n or 1
1
>>> str(n)
''
>>> int(n)
0
>>> len(n)
0
"""
def __getattr__(self, name):
return self

def __getitem__(self, item):
return self

def __call__(self, *args, **kwds):
return self

def __nonzero__(self):
return False

def __repr__(self):
return repr(None)

def __str__(self):
return ''

def __int__(self):
return 0

def __len__(self):
return 0

Source: https://github.com/tpn/tpn/blob/master/lib/tpn/util.py#L1031

Sample use: https://github.com/enversion/enversion/blob/master/lib/evn/change.py#L1300

class ChangeSet(AbstractChangeSet):
@property
def top(self):
"""
Iff one child change is present, return it.
Otherwise, return an instance of a NullObject.
"""
if self.child_count != 1:
return NullObject()
else:
top = None
for child in self:
top = child
break
return top

@property
def is_tag_create(self):
return self.top.is_tag_create

@property
def is_tag_remove(self):
return self.top.is_tag_remove

@property
def is_branch_create(self):
return self.top.is_branch_create

@property
def is_branch_remove(self):
return self.top.is_branch_remove

Having self.top potentially return a NullObject simplifies the code for
the four following properties.

> I can implement this behavior myself in pure Python, but it would be
> (a) nice to have it the in the standard library, and (b) even nicer to
> have an operator in the language, since terseness is the goal.
>

> As a motivating example: when writing web services, I often want to
> change the representation of a non-None value but also need to handle
> None gracefully. I write code like this frequently:
>
> response = json.dumps({ 'created': created.isoformat() if created
> is not None else None, 'updated': updated.isoformat() if updated
> is not None else None, ... })
>
> With a null-aware member access operator, I could write this instead:
>
> response = json.dumps({ 'created': created?.isoformat(),
> 'updated': updated?.isoformat(), ... })

If you can alter the part that creates `created` or `updated` to return
a NullObject() instead of None when applicable, you could call
`created.isoformat()` with out the addition clause.

> Thanks, Mark

Trent.
_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Andrew Barnert via Python-ideas]

On Sep 18, 2015, at 11:21, Trent Nelson <tr...@snakebite.org> wrote:
>
>> On Fri, Sep 18, 2015 at 10:42:59AM -0700, Mark Haase wrote:
>> StackOverflow has many questions
>> <http://stackoverflow.com/search?q=%5Bpython%5D+null+coalesce> on the
>> topic of null coalescing operators in Python, but I can't find any
>> discussions of them on this list or in any of the PEPs. Has the
>> addition of null coalescing operators into Python ever been discussed
>> publicly?

I believe it was raised as a side issue during other discussions (conditional expressions, exception-handling expressions, one of the pattern-matching discussions), but I personally can't remember anyone ever writing a serious proposal. I think Armin from PyPy also has a blog post mentioning the idea somewhere, as a spinoff of his arguments against PEP 484 (which turned into a more general "what's wrong with Python's type system and what could be done to fix it). One last place to look, although it'll be harder to search for, is every time people discuss whether things like dict.get are a wart on the language (because there should be a fully general way to do the equivalent) or a feature (because it's actually only useful in a handful of cases, and it's better to mark them explicitly than to try to generalize).

But my guess is that the discussion hasn't actually been had in sufficient depth to avoid having it here. (Although even if I'm right, that doesn't mean more searching isn't worth doing--to find arguments and counter arguments you may have missed, draw parallels to successes and failures in other languages, etc.) And, even if Guido hates the idea out of hand, or someone comes up with a slam-dunk argument against it, this could turn into one of those cases where it's worth someone gathering all the info and shepherding the discussion just to write a PEP for Guido to reject explicitly.

Personally, for whatever my opinion is worth (not that much), I don't have a good opinion on how it would work in Python without seeing lots of serious examples or trying it out. But I think this would be relatively easy to hack in at the tokenizer level with a quick&dirty import hook. I'll attempt it some time this weekend, in hopes that people can play with the feature. Also, it might be possible to do it less hackily with MacroPy (or it might already be part of MacroPy--often Haoyi's time machine is as good as Guido's).

>> Python has an "or" operator that can be used to coalesce false-y
>> values, but it does not have an operator to coalesce "None"
>> exclusively.
>
> Hmmm, I use this NullObject class when I want to do stuff similar to what
> you've described:

This is a very Smalltalk-y solution, which isn't a bad thing. I think having a singleton instance of NullObject (like None is a singleton instance of NoneType) so you can use is-tests, etc. might make it better, but that's arguable.

The biggest problem is that you have to write (or wrap) every API to return NullObjects instead of None, and likewise to take NullObjects. (And, if you use a PEP 484 checker, it won't understand that an optional int can hold a NullObject.)

Also, there's no way for NullObject to ensure that spam(NullObject) returns NullObject for any function spam (or, more realistically, for any function except special cases, where it's hard to define what counts as a special case but easy to understand intuitively).

And finally, there's no obvious way to make NullObject raise when you want it to raise. With syntax for nil coalescing, this is easy: ?. returns None for None, while . raises AttributeError. With separate types instead, you're putting the distinction at the point (possibly far away) where the value is produced, rather than the point where it's used.

As a side note, my experience in both Smalltalk and C# is that at some point in a large program, I'm going to end up hackily using a distinction between [nil] and nil somewhere because I needed to distinguish between an optional optional spam that "failed" at the top level vs. one that did so at the bottom level. I like the fact that in Haskell or Swift I can actually distinguish "just nil" from "nil" when I need to but usually don't have to (and the code is briefer when I don't have to), but I don't know whether that's actually essential (the [nil]) hack almost always works, and isn't that hard to read if it's used sparsely, which it almost always is).

[Guido van Rossum]

FWIW, I generally hate odd punctuation like this (@ notwithstanding) but I'm not against the idea itself -- maybe a different syntax can be invented, or maybe I could be persuaded that it's okay.

--

--Guido van Rossum (python.org/~guido)

[Andrew Barnert via Python-ideas]

On Sep 18, 2015, at 12:28, Andrew Barnert via Python-ideas <python...@python.org> wrote:
>
> Personally, for whatever my opinion is worth (not that much), I don't have a good opinion on how it would work in Python without seeing lots of serious examples or trying it out. But I think this would be relatively easy to hack in at the tokenizer level with a quick&dirty import hook. I'll attempt it some time this weekend, in hopes that people can play with the feature. Also, it might be possible to do it less hackily with MacroPy (or it might already be part of MacroPy--often Haoyi's time machine is as good as Guido's).

You can download a quick&dirty hack at https://github.com/abarnert/nonehack

This only handles the simple case of identifier?.attribute; using an arbitrary target on the left side of the . doesn't work, and there are no other none-coalescing forms like ?(...) or ?[...]. (The latter would be easy to add; the former, I don't think so.) But that's enough to handle the examples in the initial email.

So, feel free to experiment with it, and show off code that proves the usefulness of the feature.

Also, if you can think of a better syntax that will make Guido less sad, but don't know how to implement it as a hack, let me know and I'll try to do it for you.

[Mark E. Haase]

Andrew, thanks for putting together that hack. I will check it out.

Guido, you lost me at "hate odd punctuation ... maybe a different syntax can be invented". Do you mean introducing a new keyword or implementing this as a function? Or do you mean that some other punctuation might be less odd?

I'm willing to write a PEP, even if it's only purpose is to get shot down.

Mark E. Haase
202-815-0201

[Chris Angelico]

On Sat, Sep 19, 2015 at 3:42 AM, Mark Haase <meh...@gmail.com> wrote:
> StackOverflow has many questions on the topic of null coalescing operators
> in Python, but I can't find any discussions of them on this list or in any
> of the PEPs. Has the addition of null coalescing operators into Python ever
> been discussed publicly?
>
> Python has an "or" operator that can be used to coalesce false-y values, but
> it does not have an operator to coalesce "None" exclusively.

Python generally doesn't special-case None, so having a bit of magic
that works only on that one object seems a little odd. For comparison
purposes, Pike has something very similar to what you're describing,
but Pike *does* treat the integer 0 as special, so it makes good sense
there. Pike code that wants to return "a thing or NULL" will return an
object or the integer 0, where Python code will usually return an
object or None. I can't think of any situation in Python where the
language itself gives special support to None, other than it being a
keyword. You're breaking new ground.

But in my opinion, the practicality is worth it. The use of None to
represent the SQL NULL value [1], the absence of useful return value,
or other "non-values", is pretty standard. I would define the operator
pretty much the way you did above, with one exception. You say:

created?.isoformat() # is equivalent to

created.isoformat() if created is not None else None

but this means there needs to be some magic, because it should be
equally possible to write:

created?.year # equivalent to
created.year if created is not None else None

which means that sometimes it has to return None, and sometimes
(lambda *a,**ka: None). Three possible solutions:

1) Make None callable. None.__call__(*a, **ka) always returns None.
2) Special-case the immediate call in the syntax, so the equivalencies
are a bit different.
3) Add another case: func?(args) evaluates func, and if it's None,
evaluates to None without calling anything.

Option 1 would potentially mask bugs in a lot of unrelated code. I
don't think it's a good idea, but maybe others disagree.

Option 2 adds a grammatical distinction that currently doesn't exist.
When you see a nullable attribute lookup, you have to check to see if
it's a method call, and if it is, do things differently. That means
there's a difference between these:

func = obj?.attr; func()
obj?.attr()

Option 3 requires a bit more protection, but is completely explicit.
It would also have use in other situations. Personally, I support that
option; it maintains all the identities, is explicit that calling None
will yield None, and doesn't need any magic special cases. It does add
another marker, though:

created?.isoformat?() # is equivalent to
created.isoformat() if created is not None and created.isoformat is
not None else None

As to the syntax... IMO this needs to be compact, so ?. has my
support. With subscripting, should it be "obj?[idx]" or "obj[?idx]" ?
FWIW Pike uses the latter, but if C# uses the former, there's no one
obvious choice.

ChrisA

[1] Or non-value, depending on context

[Erik]

On 18/09/15 23:37, Chris Angelico wrote:
> Python generally doesn't special-case None, so having a bit of magic
> that works only on that one object seems a little odd.

So the answer here is to introduce a "magic" hook that None can make use
of (but also other classes). I can't think of an appropriate word, so
I'll use "foo" to keep it suitably abstract.

If the foo operator uses the magic method "__foo__" to mean "return an
object to be used in place of the operand should it be considered ...
false? [or some other definition - I'm not sure]" then any class can
implement that method to return an appropriate proxy object.

If that was a postfix operator which has a high precedence, then:

bar = foo?
bar.isoformat()

and the original syntax suggestion:

bar = foo?.isoformat()

... are equivalent. "?." is not a new operator. "?" is. This is
essentially a slight refinement of Chris's case 3 -

> 3) Add another case: func?(args) evaluates func, and if it's None,
> evaluates to None without calling anything.

[...]

> Option 3 requires a bit more protection, but is completely explicit.
> It would also have use in other situations. Personally, I support that
> option; it maintains all the identities, is explicit that calling None
> will yield None, and doesn't need any magic special cases. It does add
> another marker, though:

E.

[MRAB]

On 2015-09-18 23:37, Chris Angelico wrote:
> On Sat, Sep 19, 2015 at 3:42 AM, Mark Haase <meh...@gmail.com> wrote:
>> StackOverflow has many questions on the topic of null coalescing operators
>> in Python, but I can't find any discussions of them on this list or in any
>> of the PEPs. Has the addition of null coalescing operators into Python ever
>> been discussed publicly?
>>
>> Python has an "or" operator that can be used to coalesce false-y values, but
>> it does not have an operator to coalesce "None" exclusively.
>

[snip]

To me, the choice _is_ obvious: "obj?[idx]". After all, that's more in
keeping with "obj?.attr" and "func?()".

If you had "obj?[idx]", then shouldn't it also be "obj.?attr" and
"func(?)"?

[Erik]

Apologies for the self-reply. I just wanted to clarify a couple of things.

On 18/09/15 23:56, Erik wrote:
> If the foo operator uses the magic method "__foo__" to mean "return an
> object to be used in place of the operand should it be considered ...
> false? [or some other definition - I'm not sure]"

Not "false", I think. The "foo" operator is meant to mean "I will go on
to use the resulting object in any way imaginable and it must cope with
that and return a value from any attempts to use it that will generally
mean 'no'" (*).

> If that was a postfix operator which has a high precedence, then:
>
> bar = foo?
> bar.isoformat()
>
> and the original syntax suggestion:
>
> bar = foo?.isoformat()

Which is clearly wrong - the first part should be:

baz = foo?
bar = baz.isoformat()

E.

(*) Should we call the operator "shrug"?

[Sven R. Kunze]

On 19.09.2015 01:02, MRAB wrote:
> To me, the choice _is_ obvious: "obj?[idx]". After all, that's more in
> keeping with "obj?.attr" and "func?()".
>
> If you had "obj?[idx]", then shouldn't it also be "obj.?attr" and
> "func(?)"?

I agree with that.

[Sven R. Kunze]

On 19.09.2015 01:18, Erik wrote:
> Apologies for the self-reply. I just wanted to clarify a couple of
> things.
>
> On 18/09/15 23:56, Erik wrote:
>> If the foo operator uses the magic method "__foo__" to mean "return an
>> object to be used in place of the operand should it be considered ...
>> false? [or some other definition - I'm not sure]"
>
> Not "false", I think. The "foo" operator is meant to mean "I will go
> on to use the resulting object in any way imaginable and it must cope
> with that and return a value from any attempts to use it that will
> generally mean 'no'" (*).
>
>> If that was a postfix operator which has a high precedence, then:
>>
>> bar = foo?
>> bar.isoformat()
>>
>> and the original syntax suggestion:
>>
>> bar = foo?.isoformat()
>
> Which is clearly wrong - the first part should be:
>
> baz = foo?
> bar = baz.isoformat()
>
> E.
>
> (*) Should we call the operator "shrug"?

Maybe monad?

[Ryan Gonzalez]

What about "apply"? It's the closest thing to "fmap" I can think of that won't coblnfuse people...

On September 18, 2015 6:44:31 PM CDT, "Sven R. Kunze" <srk...@mail.de> wrote:

On 19.09.2015 01:18, Erik wrote:
 Apologies for the self-reply. I just wanted to clarify a couple of 
 things.

 On 18/09/15 23:56, Erik wrote:
 If the foo operator uses the magic method "__foo__" to mean "return an
 object to be used in place of the operand should it be considered ...
 false? [or some other definition - I'm not sure]"

 Not "false", I think. The "foo" operator is meant to mean "I will go 
 on to use the resulting object in any way imaginable and it must cope 
 with that and return a value from any attempts to use it that will 
 generally mean 'no'" (*).

 If that was a postfix operator which has a high precedence, then:

 bar = foo?
 bar.isoformat()

 and the original syntax suggestion:

 bar = foo?.isoformat()

 Which is clearly wrong - the first part should be:

 baz = foo?
 bar = baz.isoformat()

 E.

 (*) Should we call the operator "shrug"?

Maybe monad?

Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.

[Sven R. Kunze]

On 19.09.2015 01:47, Ryan Gonzalez wrote:
> What about "apply"? It's the closest thing to "fmap" I can think of
> that won't coblnfuse people...

Are you sure? I think "maybe" better reflects the purpose of "?".


Nevertheless, I would love to see support for the maybe monad in Python.

Best,
Sven
_______________________________________________

[Joseph Jevnik]

Is there a reason that this needs explicit support, it is trivial to implement maybe in pure python.

[MRAB]

On 2015-09-19 00:44, Sven R. Kunze wrote:
>
>
> On 19.09.2015 01:18, Erik wrote:
>> Apologies for the self-reply. I just wanted to clarify a couple of
>> things.
>>
>> On 18/09/15 23:56, Erik wrote:
>>> If the foo operator uses the magic method "__foo__" to mean "return an
>>> object to be used in place of the operand should it be considered ...
>>> false? [or some other definition - I'm not sure]"
>>
>> Not "false", I think. The "foo" operator is meant to mean "I will go
>> on to use the resulting object in any way imaginable and it must cope
>> with that and return a value from any attempts to use it that will
>> generally mean 'no'" (*).
>>
>>> If that was a postfix operator which has a high precedence, then:
>>>
>>> bar = foo?
>>> bar.isoformat()
>>>
>>> and the original syntax suggestion:
>>>
>>> bar = foo?.isoformat()
>>
>> Which is clearly wrong - the first part should be:
>>
>> baz = foo?
>> bar = baz.isoformat()
>>
>> E.
>>
>> (*) Should we call the operator "shrug"?
>
> Maybe monad?
>

Too fancy.

How about "ni"? :-)

[Ryan Gonzalez]

On September 18, 2015 6:58:23 PM CDT, "Sven R. Kunze" <srk...@mail.de> wrote:
>On 19.09.2015 01:47, Ryan Gonzalez wrote:
>> What about "apply"? It's the closest thing to "fmap" I can think of
>> that won't coblnfuse people...
>
>Are you sure? I think "maybe" better reflects the purpose of "?".
>

That's better. Or "optional".

>
>Nevertheless, I would love to see support for the maybe monad in
>Python.
>
>Best,
>Sven

--
Sent from my Nexus 5 with K-9 Mail. Please excuse my brevity.

[Akira Li]

Ryan Gonzalez <rym...@gmail.com> writes:

>>On 19.09.2015 01:18, Erik wrote:

...

>>>
>>> baz = foo?
>>> bar = baz.isoformat()
>>>
>>> E.
>>>
>>> (*) Should we call the operator "shrug"?
>>
>>Maybe monad?

http://stackoverflow.com/questions/8507200/maybe-kind-of-monad-in-python


_______________________________________________

[Andrew Barnert via Python-ideas]

On Sep 18, 2015, at 15:56, Erik <pyt...@lucidity.plus.com> wrote:
>
>> On 18/09/15 23:37, Chris Angelico wrote:
>> Python generally doesn't special-case None, so having a bit of magic
>> that works only on that one object seems a little odd.
>
> So the answer here is to introduce a "magic" hook that None can make use of (but also other classes). I can't think of an appropriate word, so I'll use "foo" to keep it suitably abstract.
>
> If the foo operator uses the magic method "__foo__" to mean "return an object to be used in place of the operand should it be considered ... false? [or some other definition - I'm not sure]" then any class can implement that method to return an appropriate proxy object.
>
> If that was a postfix operator which has a high precedence, then:
>
> bar = foo?
> bar.isoformat()
>
> and the original syntax suggestion:
>
> bar = foo?.isoformat()
>
> ... are equivalent. "?." is not a new operator. "?" is. This is essentially a slight refinement of Chris's case 3 -

I like this (modulo the corrections later in the thread). It's simpler and more flexible than the other options, and also comes closer to resolving the "spam?.eggs" vs. "spam?.cheese()" issue, by requiring "spam?.cheese?()".

Obviously "spam?" returns something with a __getattr__ method that just passes through to spam.__getattr__, except that on NoneType it returns something with a __getattr__ that always returns None. That solves the eggs case.

Next, "spam?.cheese?" returns something with a __call__ method that just passed through to spam?.cheese.__call__, except that on NoneType it returns something with a __call__ that always returns None. That solves the cheese case.

If you make None? return something whose other dunder methods also return None (except for special cases like __repr__), this also gives you "spam ?+ 3". (I'm not sure if that's a good thing or a bad thing...) Of course there's no way to do "spam ?= 3" (but I'm pretty sure that's a good thing).

So, do we need a dunder method for the "?" operator? What else would you use it for besides None?

[Random832]

On Fri, Sep 18, 2015, at 18:37, Chris Angelico wrote:
> created?.isoformat?() # is equivalent to
> created.isoformat() if created is not None and created.isoformat is
> not None else None

More or less - it'd only look up the attribute once.

> As to the syntax... IMO this needs to be compact, so ?. has my
> support. With subscripting, should it be "obj?[idx]" or "obj[?idx]" ?
> FWIW Pike uses the latter, but if C# uses the former, there's no one
> obvious choice.

?[ has the benefit of being consistent with ?. - and ?(, for that
matter. It actually suggests a whole range of null-coalescing operators.
?* for multiply? A lot of these things are done already by the normal
operators for statically-typed nullable operands in C#.

That could get hairy fast - I just thought of a radical alternative
that I'm not even sure if I support: ?(expr) as a lexical context that
changes the meaning of all operators.

[Chris Angelico]

On Sat, Sep 19, 2015 at 10:49 AM, Andrew Barnert <abar...@yahoo.com> wrote:
> Obviously "spam?" returns something with a __getattr__ method that just passes through to spam.__getattr__, except that on NoneType it returns something with a __getattr__ that always returns None. That solves the eggs case.
>
> Next, "spam?.cheese?" returns something with a __call__ method that just passed through to spam?.cheese.__call__, except that on NoneType it returns something with a __call__ that always returns None. That solves the cheese case.
>

Hang on, how do you do this? How does the operator know the difference
between "spam?", which for None has to have __getattr__ return None,
and "spam?.cheese?" that returns (lambda: None)?

ChrisA

[Andrew Barnert via Python-ideas]

On Sep 18, 2015, at 16:58, Sven R. Kunze <srk...@mail.de> wrote:
>
>> On 19.09.2015 01:47, Ryan Gonzalez wrote:
>> What about "apply"? It's the closest thing to "fmap" I can think of that won't coblnfuse people...
>
> Are you sure? I think "maybe" better reflects the purpose of "?".
>
>
> Nevertheless, I would love to see support for the maybe monad in Python.

I think this, and the whole discussion of maybe and fmap, is off the mark here.

It's trivial to create a maybe type in Python.

What's missing is the two things that make it useful: (1) pattern matching, and (2) a calling syntax and a general focus on HOFs that make fmap natural. Without at least one of those, maybe isn't useful. And adding either of those to Python is a huge proposal, much larger than null coalescing, and a lot less likely to gain support.

Also, the monadic style of failure propagation directly competes with the exception-raising style, and they're both contagious. A well-designed language and library can have both side by side if it, e.g., rigorously restricts exceptions to only truly exceptional cases, but the boat for that sailed decades ago in Python. So just having them side by side would lead to the exact same problems as C++ code that mixes exception-based and status-code-based APIs, or JavaScript code that mixes exceptions and errbacks or promise.fail handlers.

Personally, whenever I think to myself "I could really use maybe here" in some Python code, that's a sign that I'm not thinking Pythonically, and either need to switch gears in my brain or switch languages. Just like when I start thinking about how I could get rid of that with statement with an RAII class, and maybe an implicit conversion operator....

[Andrew Barnert via Python-ideas]

On Sep 18, 2015, at 18:00, Chris Angelico <ros...@gmail.com> wrote:
>
>> On Sat, Sep 19, 2015 at 10:49 AM, Andrew Barnert <abar...@yahoo.com> wrote:
>> Obviously "spam?" returns something with a __getattr__ method that just passes through to spam.__getattr__, except that on NoneType it returns something with a __getattr__ that always returns None. That solves the eggs case.
>>
>> Next, "spam?.cheese?" returns something with a __call__ method that just passed through to spam?.cheese.__call__, except that on NoneType it returns something with a __call__ that always returns None. That solves the cheese case.
>
> Hang on, how do you do this? How does the operator know the difference
> between "spam?", which for None has to have __getattr__ return None,
> and "spam?.cheese?" that returns (lambda: None)?

>>> spam
None
>>> spam?
NoneQuestion
>>> spam?.cheese
None
>>> spam?.cheese?
NoneQuestion
>>> spam?.cheese?()
None

All you need to make this work is:

* "spam?" returns NoneQuestion if spam is None else spam
* NoneQuestion.__getattr__(self, *args, **kw) returns None.
* NoneQuestion.__call__(self, *args, **kw) returns None.

Optionally, you can add more None-returning methods to NoneQuestion. Also, whether NoneQuestion is a singleton, has an accessible name, etc. are all bikesheddable.

I think it's obvious what happens is "spam" is not None and "spam.cheese" is, or of both are None, but if not, I can work them through as well.

[MRAB]

On 2015-09-19 02:10, Andrew Barnert via Python-ideas wrote:
> On Sep 18, 2015, at 18:00, Chris Angelico <ros...@gmail.com> wrote:
>>
>>> On Sat, Sep 19, 2015 at 10:49 AM, Andrew Barnert <abar...@yahoo.com> wrote:
>>> Obviously "spam?" returns something with a __getattr__ method that just passes through to spam.__getattr__, except that on NoneType it returns something with a __getattr__ that always returns None. That solves the eggs case.
>>>
>>> Next, "spam?.cheese?" returns something with a __call__ method that just passed through to spam?.cheese.__call__, except that on NoneType it returns something with a __call__ that always returns None. That solves the cheese case.
>>
>> Hang on, how do you do this? How does the operator know the difference
>> between "spam?", which for None has to have __getattr__ return None,
>> and "spam?.cheese?" that returns (lambda: None)?
>
>>>> spam
> None
>>>> spam?
> NoneQuestion
>>>> spam?.cheese
> None
>>>> spam?.cheese?
> NoneQuestion
>>>> spam?.cheese?()
> None
>
> All you need to make this work is:
>
> * "spam?" returns NoneQuestion if spam is None else spam
> * NoneQuestion.__getattr__(self, *args, **kw) returns None.
> * NoneQuestion.__call__(self, *args, **kw) returns None.
>
> Optionally, you can add more None-returning methods to NoneQuestion. Also, whether NoneQuestion is a singleton, has an accessible name, etc. are all bikesheddable.
>
> I think it's obvious what happens is "spam" is not None and "spam.cheese" is, or of both are None, but if not, I can work them through as well.
>

I see it as "spam? doing "Maybe(spam)" and then attribute access
checking returning None if the wrapped object is None and getting the
attribute from it if not.

I think that the optimiser could probably avoid the use of Maybe in
cases like "spam?.cheese".

[MRAB]

I've thought of another issue:

If you write "spam?(sing_lumberjack_song())", won't it still call
sing_lumberjack_song even if spam is None? After all, Python evaluates
the arguments before looking up the call, so it won't know that "spam"
is None until it tries to call "spam?".

That isn't a problem with "spam.sing_lumberjack_song() if spam is not
None else None" or if it's optimised to that, but "m = spam?;
m(sing_lumberjack_song())" is a different matter.

perhaps a "Maybe" object should also support "?" so you could write "m
= spam?; m?(sing_lumberjack_song())". "Maybe" could be idempotent, so
"Maybe(Maybe(x))" returns the same result as "Maybe(x)".

[Mark E. Haase]

Andrew, I really like that idea. Turning back to the null coalescing operator (spelled ?? in other languages), how do you think that fits in? 

Consider this syntax:

>>> None? or 1

1

This works if NoneQuestion overrides __nonzero__ to return False.

>>> 0? or 1

0

This doesn't work, because 0? returns 0, and "0 or 1" is 1. 

We could try this instead, if NoneQuestion overrides __or__:

>>> 0? | 1

0

>>> 0 ?| 1

0

This looks a little ugly, and it would be nice (as MRAB pointed out) if null coalescing short circuited.

>>> None? or None?

This also doesn't work quite right. If both operands are None, we want the expression to evaluate to None, not NoneQuestion. Should null coalescing be a separate operator? And if so, are "?" and "??" too similar?

Can anybody think of realistic use cases for overriding a magic method for the "?" operator? I would like to include such use cases in a PEP. One possible use case: being able to coalesce empty strings.

>>> s1 = MyString('')

>>> s2 = MyString('foobar')

>>> s1? or s2

MyString('foobar')

[Chris Angelico]

On Sat, Sep 19, 2015 at 12:06 PM, Mark E. Haase <meh...@gmail.com> wrote:
> Can anybody think of realistic use cases for overriding a magic method for
> the "?" operator? I would like to include such use cases in a PEP. One
> possible use case: being able to coalesce empty strings.
>
>>>> s1 = MyString('')
>>>> s2 = MyString('foobar')
>>>> s1? or s2
> MyString('foobar')

Frankly, I think this is a bad idea. You're potentially coalescing
multiple things with the same expression, and we already have a way of
spelling that: the "or" operator. If you don't want a generic "if it's
false, use this", and don't want a super-specific "if it's None, use
this", then how are you going to define what it is? And more
importantly, how do you reason about the expression "s1? or s2"
without knowing exactly what types coalesce to what? Let's keep the
rules simple. Make this a special feature of the None singleton, and
all other objects simply return themselves - for the same reason that
a class isn't allowed to override the "is" operator.

[Andrew Barnert via Python-ideas]

You're right; I didn't think about that. But I don't think that's a problem.

I believe C#, Swift, etc. all evaluate the arguments in their equivalent. And languages like ObjC that do automatic nil coalescing for all method calls definitely evaluate them. If you really want to switch on spam and not call sing_lumberjack_song, you can always do that manually, right?

> perhaps a "Maybe" object should also support "?" so you could write "m
> = spam?; m?(sing_lumberjack_song())". "Maybe" could be idempotent, so
> "Maybe(Maybe(x))" returns the same result as "Maybe(x)".

That actually makes sense just for its own reasons.

Actually, now that I think about it, the way I defined it above already gives you think: if spam? is spam if it's anything but None, then spam?? is always spam?, right?

[Andrew Barnert via Python-ideas]

On Sep 18, 2015, at 19:06, Mark E. Haase <meh...@gmail.com> wrote:

Andrew, I really like that idea. Turning back to the null coalescing operator (spelled ?? in other languages), how do you think that fits in? 

Consider this syntax:

>>> None? or 1

I don't think there's any easy way to make "spam? or 1" work any better than "spam or 1" already does, partly for the reasons you give below, but also because it doesn't seem to fit the design in any obvious way.

I guess that means postix ? doesn't quite magically solve everything...

This also doesn't work quite right. If both operands are None, we want the expression to evaluate to None, not NoneQuestion. Should null coalescing be a separate operator? And if so, are "?" and "??" too similar?

As MRAB pointed out, there seem to be good reasons to let spam?? mean the same thing as spam? (and that follows automatically from the simplest possible definition, the one I gave above). So I think "spam ?? eggs" is ambiguous between the postfix operator and the infix operator without lookahead, at least to a human, and possibly to the compiler as well.

I suppose ?: as in ColdFusion might work, but (a) ewwww, (b) it regularly confuses novices to CF, and (c) it's impossible to search for, because ?: no matter how you quote it gets you the C ternary operator....

Can anybody think of realistic use cases for overriding a magic method for the "?" operator? I would like to include such use cases in a PEP. One possible use case: being able to coalesce empty strings.

>>> s1 = MyString('')

>>> s2 = MyString('foobar')

>>> s1? or s2

MyString('foobar')

This seems like a bad idea. Empty strings are already falsey. If you want this behavior, why not just use "s1 or s2", which already works, and for obvious reasons?

[Steven D'Aprano]

On Sat, Sep 19, 2015 at 12:02:42AM +0100, MRAB wrote:

> To me, the choice _is_ obvious: "obj?[idx]". After all, that's more in
> keeping with "obj?.attr" and "func?()".
>
> If you had "obj?[idx]", then shouldn't it also be "obj.?attr" and
> "func(?)"?

No.

If I understand the idea, obj?.attr returns None if obj is None,
otherwise returns obj.attr. The question mark (shrug operator?) applies
to `obj` *before* the attribute lookup, so it should appear *before* the
dot (since we read from left-to-right).

The heuristic for remembering the order is that the "shrug" (question
mark) operator applies to obj, so it is attached to obj, before any
subsequent operation.

For the sake of brevity, using @ as a placeholder for one of attribute
access, item/key lookup, or function call, then we have:

obj?@

as syntactic sugar for:

None if obj is None else obj@

Furthermore, we should be able to chain a sequence of such @s:

paperboy.receive(customer?.trousers.backpocket.wallet.extract(2.99))


being equivalent to:

paperboy.receive(None if customer is None else
customer.trousers.backpocket.wallet.extract(2.99)
)


Let's just assume we have a good reason for chaining lookups that isn't
an egregious violation of the Law of Demeter, and not get into a debate
over OOP best practices, okay? :-)

Suppose that wallet itself may also be None. Then we can easily deal
with that situation too:

paperboy.receive(customer?.trousers.backpocket.wallet?.extract(2.99))


which I think is a big win over either of these two alternatives:

# 1
paperboy.receive(None if customer is None else
None if customer.trousers.backpocket.wallet is None
else customer.trousers.backpocket.wallet.extract(2.99)
)

# 2
if customer is not None:
wallet = customer.trousers.backpocket.wallet
if wallet is not None:
paperboy.receive(wallet.extract(2.99))


It's a funny thing, I'm usually not a huge fan of symbols outside of
maths operators, and I strongly dislike the C ? ternary operator, but
this one feels really natural to me. I didn't have even the most
momentary "if you want Perl, you know where to find it" thought.



--
Steve

[Ryan Gonzalez]

This is likely going to get shot down quickly...

I know CoffeeScript is not regarded too well in this community (well, at least based on Guido's remarks on parsing it), but what if x? was shorthand for x is None? In CS, it's called the existential operator.

[Guido van Rossum]

On Fri, Sep 18, 2015 at 8:41 PM, Steven D'Aprano <st...@pearwood.info> wrote:

It's a funny thing, I'm usually not a huge fan of symbols outside of
maths operators, and I strongly dislike the C ? ternary operator, but
this one feels really natural to me. I didn't have even the most
momentary "if you want Perl, you know where to find it" thought.

I do, but at least the '?' is part of an operator, not part of the name (as it is in Ruby?).

I really, really, really don't like how it looks, but here's one thing: the discussion can be cut short and focus almost entirely on whether this is worth making Python uglier (and whether it's even ugly :-). The semantics are crystal clear and it's obvious that the way it should work is by making "?.", ?(" and "?[" new operators or operator pairs -- the "?" should not be a unary postfix operator but a symbol that combines with certain other symbols.

Let me propose a (hyper?)generalization: it could be combined with any binary operation, e.g. "a?+b" would mean "None if a is None else a+b". Sadly (as hypergeneralizations tend to do?) this also leads to a negative observation: what if I wanted to write "None if b is None else a+b"? (And don't be funny and say I should swap a and b -- they could be strings.) Similar for what if you wanted to do this with a unary operator, e.g. None if x is None else -x. Maybe we could write "a+?b" and "-?x"? But I don't think the use cases warrant these much.

Finally, let's give it a proper name -- let's call it the uptalk operator.

--

--Guido van Rossum (python.org/~guido)

[Steven D'Aprano]

On Fri, Sep 18, 2015 at 05:49:36PM -0700, Andrew Barnert via Python-ideas wrote:

> Obviously "spam?" returns something with a __getattr__ method that
> just passes through to spam.__getattr__, except that on NoneType it
> returns something with a __getattr__ that always returns None. That
> solves the eggs case.

Ah, and now my enthusiasm for the whole idea is gone...

In my previous response, I imagined spam?.attr to be syntactic sugar for
`None if spam is None else spam.attr`. But having ? be an ordinary
operator that returns a special Null object feels too "Design Pattern-y"
to me. I think the Null design pattern is actually harmful, and I would
not like to see this proposal implemented this way.

(In another email, Andrew called the special object something like
NoneMaybe or NoneQuestion, I forget which. I'm going to call the object
Null, since that's less typing.)

The Null object pattern sounds like a great idea at first, but I find it
to be a code smell at best and outright harmful at worst. If you are
passing around an object which is conceptually None, but unlike None
reliably does nothing without raising an exception no matter what you do
with it, that suggests to me that something about your code is not
right.

If your functions already accept None, then you should just use None. If
they don't accept None, then why are you trying to smuggle None into
them using a quasi-None that unconditionally hides errors?

Here are some problems with the Null pattern as I see it:

(1) Suppose that spam? returns a special Null object, and Null.attr
itself returns Null. (As do Null[item] and Null(arg), of course.) This
matches the classic Null object design pattern, and gives us chaining
for free:

value = obj?.spam.eggs.cheese

But now `value` is Null, which may not be what we expect and may in fact
be a problem if we're expecting it to be "an actual value, or None"
rather than our quasi-None Null object.

Because `value` is now a Null, every time we pass it to a function, we
risk getting new Nulls in places that shouldn't get them. If a function
isn't expecting None, we should get an exception, but Null is designed
to not raise exceptions no matter what you do with it. So we risk
contaminating our data with Nulls in unexpected places.

Eventually, of course, there comes a time where we need to deal with the
actual value. With the Null pattern in place, we have to deal with two
special cases, not one:

# I assume Null is a singleton, otherwise use isinstance
if filename is not None and filename is not Null:
os.unlink(filename)

A small nuisance, to be sure, but part of the reason why I really don't
think much of the Null object pattern. It sounds good on paper, but I
think it's actually more dangerous and inconvenient than the problem it
tries to solve.


(2) We can avoid the worst of the Null design (anti-)pattern by having
Null.attr return None instead of Null. Unfortunately, that means we've
lost automatic chaining. If you have an object that might be None, we
have to explicitly use the ? operator after each lookup except the last:

value = obj?.spam?.eggs?.cheese

which is (a) messy, (b) potentially inefficient, and (c) potentially
hides subtle bugs.

Here is a scenario where it hides bugs. Suppose obj may be None, but if
it is not, then obj.spam *must* be a object with an eggs attribute. If
obj.spam is None, that's a bug that needs fixing. Suppose we start off
by writing the obvious thing:

obj?.spam.eggs

but that fails because obj=None raises an exception:

obj? returns Null
Null.spam returns None
None.eggs raises

So to protect against that, we might write:

obj?.spam?.eggs

but that protects against too much, and hides the fact that obj.spam
exists but is None.

As far as I am concerned, any use of a Null object has serious
downsides. If people want to explicitly use it in their own code, well,
good luck with that. I don't think Python should be making it a
built-in.

I think the first case, the classic Null design pattern, is actually
*better* because the downsides are anything but subtle, and people will
soon learn not to touch it with a 10ft pole *wink*, while the second
case, the "Null.attr gives None" case, is actually worse because it
isn't *obviously* wrong and can subtly hide bugs.


How does my earlier idea of ? as syntactic sugar compare with those?

In that case, there is no special Null object, there's only None. So we
avoid the risk of Null infection, and avoid needing to check specially
for Null. It also avoids the bug-hiding scenario:

obj?.spam.eggs.cheese

is equivalent to:

None if obj is None else obj.spam.eggs


If obj is None, we get None, as we expect. If it is not None, we get
obj.spam.eggs as we expect. If obj.spam is wrongly None, then we get an
exception, as we should.



--
Steve

[Stephen J. Turnbull]

Andrew Barnert via Python-ideas writes:

> So, do we need a dunder method for the "?" operator? What else
> would you use it for besides None?

NaNs in a pure-Python implementation of float or Decimal. (This is
not a practical suggestion.)

A true SQL NULL type. It's always bothered me that most ORMs map NULL
to None but there are plenty of other ways to inject None into a
Python computation. (This probably isn't a practical suggestion
either unless Random832's suggestion of ?() establishing a lexical
context were adopted.)

The point is that Maybe behavior is at least theoretically useful in
subcategories, with special objects other than None.

Sven's suggestion of calling this the "monad" operator triggers a
worry in me, however. In Haskell, the Monad type doesn't enforce the
monad laws, only the property of being an endofunctor. That
apparently turns out to be enough in practice to make the Monad type
very useful. However, in Python we have no way to enforce that
property. I don't have the imagination to come up with a truly
attractive nuisance here, and this operator doesn't enable general
functorial behavior, so maybe it's not a problem.

[Random832]

Guido van Rossum <gu...@python.org> writes:
> Let me propose a (hyper?)generalization: it could be combined with any
> binary operation, e.g. "a?+b" would mean "None if a is None else a+b".

I'd have read it as "None if a is None or b is None else a+b". If you
want to only do it for one of the operands you should be explicit.

I'm not sure if I have a coherent argument for why this shouldn't apply
to ?[, though.

[C Anthony Risinger]

On Fri, Sep 18, 2015 at 11:21 PM, Guido van Rossum <gu...@python.org> wrote:

On Fri, Sep 18, 2015 at 8:41 PM, Steven D'Aprano <st...@pearwood.info> wrote:

It's a funny thing, I'm usually not a huge fan of symbols outside of
maths operators, and I strongly dislike the C ? ternary operator, but
this one feels really natural to me. I didn't have even the most
momentary "if you want Perl, you know where to find it" thought.

I do, but at least the '?' is part of an operator, not part of the name (as it is in Ruby?).

I really, really, really don't like how it looks, but here's one thing: the discussion can be cut short and focus almost entirely on whether this is worth making Python uglier (and whether it's even ugly :-). The semantics are crystal clear and it's obvious that the way it should work is by making "?.", ?(" and "?[" new operators or operator pairs -- the "?" should not be a unary postfix operator but a symbol that combines with certain other symbols.

I really liked this whole thread, and I largely still do -- I?think -- but I'm not sure I like how `?` suddenly prevents whole blocks of code from being evaluated. Anything within the (...) or [...] is now skipped (IIUC) just because a `?` was added, which seems like it could have side effects on the surrounding state, especially since I expect people will use it for squashing/silencing or as a convenient trick after the fact, possibly in code they did not originally write.

If the original example included a `?` like so:

    response = json.dumps?({

        'created': created?.isoformat(),

        'updated': updated?.isoformat(),

        ...

    })

should "dumps" be None, the additional `?` (although though you can barely see it) prevents *everything else* from executing. This may cause confusion about what is being executed, and when, especially once nesting (to any degree really) and/or chaining comes into play!

Usually when I want to use this pattern, I find I just need to write things out more. The concept itself vaguely reminds me of PHP's use of `@` for squashing errors. In my opinion, it has some utility but has too much potential impact on program flow without being very noticeable. If I saw more than 1 per line, or a couple within a few lines, I think my ability to quickly identify -> analyze -> comprehend possible routes in program control flow decreases. I feel like I'll fault more, double back, and/or make sure I forevermore look harder for sneaky `?`s.

I probably need to research more examples of how such a thing is used in real code, today. This will help me get a feel for how people might want to integrate the new `?` capability into their libraries and apis, maybe that will ease my readability reservations.

Thanks,

--

C Anthony

[Greg Ewing]

Guido van Rossum wrote:

> Finally, let's give it a proper name -- let's call it the uptalk operator.

Um... why? Is this Monty reference I'm missing?

--
Greg

[Sven R. Kunze]

On 19.09.2015 07:14, Stephen J. Turnbull wrote:

A true SQL NULL type.  It's always bothered me that most ORMs map NULL
to None but there are plenty of other ways to inject None into a
Python computation.  (This probably isn't a practical suggestion
either unless Random832's suggestion of ?() establishing a lexical
context were adopted.)

I definitely agree here. Internally, we have a guideline telling us to avoid None or NULL whenever possible. Andrew's remark about 'code smell' is definitely appropriate.


There was a great discussion some years ago on one of the RDF semantics mailing list about the semantics of NULL (in RDF). It turned out to have 6 or 7 semantics WITHOUT any domain-specific focus (don't know, don't exists, is missing, etc. -- can't remember all of them). I feel that is one reason why Python programs should avoid None: we don't guess.

The point is that Maybe behavior is at least theoretically useful in
subcategories, with special objects other than None.

Sven's suggestion of calling this the "monad" operator triggers a
worry in me, however.  In Haskell, the Monad type doesn't enforce the
monad laws, only the property of being an endofunctor.  That
apparently turns out to be enough in practice to make the Monad type
very useful.  However, in Python we have no way to enforce that
property.  I don't have the imagination to come up with a truly
attractive nuisance here, and this operator doesn't enable general
functorial behavior, so maybe it's not a problem.

Sleeping one night over it, I now tend to change my mind regarding this. Maybe, it's better to DEAL with None as in remove them from the code, from the database, from the YAML files and so forth instead of making it easier to work with them. Restricting oneself, would eventually lead to more predictable designs.

Does this makes sense somehow?


Issue is, None is so convenient to work with. You only find out the code smell when you discover a "NoneType object does not have attribute X" exception some months later and start looking where the heck the None could come from. What can we do here?

[Steven D'Aprano]

On Sat, Sep 19, 2015 at 03:17:07AM -0500, C Anthony Risinger wrote:

> I really liked this whole thread, and I largely still do -- I?think -- but
> I'm not sure I like how `?` suddenly prevents whole blocks of code from
> being evaluated. Anything within the (...) or [...] is now skipped (IIUC)
> just because a `?` was added, which seems like it could have side effects
> on the surrounding state, especially since I expect people will use it for
> squashing/silencing or as a convenient trick after the fact, possibly in
> code they did not originally write.

I don't think this is any different from other short-circuiting
operators, particularly `and` and the ternary `if` operator:

result = obj and obj.method(expression)

result = obj.method(expression) if obj else default

In both cases, `expression` is not evaluated if obj is falsey. That's
the whole point.

> If the original example included a `?` like so:
>
> response = json.dumps?({
> 'created': created?.isoformat(),
> 'updated': updated?.isoformat(),
> ...
> })
>
> should "dumps" be None, the additional `?` (although though you can barely
> see it) prevents *everything else* from executing.

We're still discussing the syntax and semantics of this, so I could be
wrong, but my understanding of this is that the *first* question mark
prevents the expressions in the parens from being executed:

json.dumps?( ... )

evaluates as None if json.dumps is None, otherwise it evaluates the
arguments and calls the dumps object. In other words, rather like this:

_temp = json.dumps # temporary value
if _temp is None:
response = None
else:
response = _temp({
'created': None if created is None else created.isoformat(),
'updated': None if updated is None else updated.isoformat(),
...
})
del _temp


except the _temp name isn't actually used. The whole point is to avoid
evaluating an expression (attribute looking, index/key lookup, function
call) which will fail if the object is None, and if you're not going to
call the function, why evaluate the arguments to the function?

> This may cause confusion
> about what is being executed, and when, especially once nesting (to any
> degree really) and/or chaining comes into play!

Well, yes, people can abuse most any syntax.

> Usually when I want to use this pattern, I find I just need to write things
> out more. The concept itself vaguely reminds me of PHP's use of `@` for
> squashing errors.

I had to look up PHP's @ and I must say I'm rather horrified. According
to the docs, all it does is suppress the error reporting, it does
nothing to prevent or recover from errors. There's not really an
equivalent in Python, but I suppose this is the closest:

# similar to PHP's $result = @(expression);
try:
result = expression
except:
result = None


This is nothing like this proposal. It doesn't suppress arbitrary
errors. It's more like a conditional:

# result = obj?(expression)
if obj is None:
result = None
else:
result = obj(expression)


If `expression` raises an exception, it will still be raised, but only
if it is actually evaluated, just like anything else protected by an
if...else or short-circuit operator.

[Stephen J. Turnbull]

Sven R. Kunze writes:

> Issue is, None is so convenient to work with. You only find out the
> code smell when you discover a "NoneType object does not have
> attribute X"

That's exactly what should happen (analogous to a "signalling NaN").
The problem is if you are using None as a proxy for a NULL in another
subsystem that has "NULL contagion" (I prefer that to "coalescing").

At this point the thread ends for me because I'm not going try to tell
the many libraries that have chosen to translate NULL to None and vice
versa that they are wrong.

[Guido van Rossum]

"Uptalk" is an interesting speech pattern where every sentence sounds like a question. Google it, there's some interesting research.

The "null pattern" is terrible. Uptalk should not be considered a unary operator that returns a magical value. It's a modifier on other operators (somewhat similar to the way "+=" and friends are formed).

In case someone missed it, uptalk should test for None, not for a falsey value.

I forgot to think about the scope of the uptalk operator (i.e. what is skipped when it finds a None). There are some clear cases (the actual implementation should avoid double evaluation of the tested expression, of course):

  a.b?.c.d[x, y](p, q) === None if a.b is None else a.b.c.d[x, y](p, q)

  a.b?[x, y].c.d(p, q) === None if a.b is None else a.b[x, y].c.d(p, q)

  a.b?(p, q).c.d[x, y] === None if a.b is None else a.b(p, q).c.d[x, y]

But what about its effect on other operators in the same expression? I think this is reasonable:

  a?.b + c.d === None if a is None else a.b + c.d

OTOH I don't think it should affect shortcut boolean operators (and, or):

  a?.b or x === (None if a is None else a.b) or x

It also shouldn't escape out of comma-separated lists, argument lists, etc.:

  (a?.b, x) === ((None if a is None else a.b), x)

  f(a?.b) === f((None if a is None else a.b))

Should it escape from plain parentheses? Which of these is better?

  (a?.b) + c === (None if a is None else a.b) + c    # Fails unless c overloads None+c

  (a?.b) + c === None if a is None else (a.b) + c    # Could be surprising if ? is deeply nested

Here are some more edge cases / hypergeneralizations:

  {k1?: v1, k2: v2} === {k2: v2} if k1 is None else {k1: v1, k2: v2}   # ?: skips if key is None

  # But what to do to skip None values?

Could we give ?= a meaning in assignment, e.g. x ?= y could mean:

  if y is not None:

      x = y

More fun: x ?+= y could mean:

  if x is None:

      x = y

  elif y is not None:

      y += y

You see where this is going. Downhill fast. :-)

[Chris Angelico]

On Sun, Sep 20, 2015 at 2:21 AM, Guido van Rossum <gu...@python.org> wrote:
> Should it escape from plain parentheses? Which of these is better?
>
> (a?.b) + c === (None if a is None else a.b) + c # Fails unless c
> overloads None+c
> (a?.b) + c === None if a is None else (a.b) + c # Could be surprising
> if ? is deeply nested

My recommendation: It should _not_ escape. That way, you get control
over how far out the Noneness goes - you can bracket it in as tight as
you like.

ChrisA

[MRAB]

It shouldn't escape beyond anything having a lower precedence.

> Here are some more edge cases / hypergeneralizations:
>
> {k1?: v1, k2: v2} === {k2: v2} if k1 is None else {k1: v1, k2: v2}
> # ?: skips if key is None
> # But what to do to skip None values?
>
> Could we give ?= a meaning in assignment, e.g. x ?= y could mean:
>
> if y is not None:
> x = y
>

Shouldn't that be:

if x is not None:
x = y

? It's the value before the '?' that's tested.

> More fun: x ?+= y could mean:
>
> if x is None:
> x = y
> elif y is not None:
> y += y
>

Or:

if x is None:
pass
else:
x += y

> You see where this is going. Downhill fast. :-)
>

Could it be used postfix:

a +? b === None if b is None else a + b

-?a === None if a is None else -a

or both prefix and postfix:

a ?+? b === None if a is None or b is None else a + b

?

[Sven R. Kunze]

On 19.09.2015 14:48, Stephen J. Turnbull wrote:
> Sven R. Kunze writes:
>
> > Issue is, None is so convenient to work with. You only find out the
> > code smell when you discover a "NoneType object does not have
> > attribute X"
>
> That's exactly what should happen (analogous to a "signalling NaN").

Not my point, Stephen. My point is, you better avoid None (despite its
convenience) because you are going to have a hard time finding its
origin later in the control flow.

Question still stands: is None really necessary to justify the
introduction of convenience operators like "?." etc.?

> The problem is if you are using None as a proxy for a NULL in another
> subsystem that has "NULL contagion" (I prefer that to "coalescing").

How would you solve instead?


Best,
Sven

[Ryan Gonzalez]

I think the core issue is that, whether or not it should be used, APIs already return None values, so a convenience operator might as well be added.

On September 19, 2015 2:09:48 PM CDT, "Sven R. Kunze" <srk...@mail.de> wrote:

On 19.09.2015 14:48, Stephen J. Turnbull wrote:
 Sven R. Kunze writes:

 Issue is, None is so convenient to work with. You only find out the
 code smell when you discover a "NoneType object does not have
 attribute X"

 That's exactly what should happen (analogous to a "signalling NaN").

Not my point, Stephen. My point is, you better avoid None (despite its 
convenience) because you are going to have a hard time finding its 
origin later in the control flow.

Question still stands: is None really necessary to justify the 
introduction of convenience operators like "?." etc.?

 The problem is if you are using None as a proxy for a NULL in another
 subsystem that has "NULL contagion" (I prefer that to "coalescing").

How would you solve instead?


Best,
Sven

Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Xavier Combelle]

2015-09-19 21:24 GMT+02:00 Ryan Gonzalez <rym...@gmail.com>:

I think the core issue is that, whether or not it should be used, APIs already return None values, so a convenience operator might as well be added.

I'm curious on which API returning None, a major bonus on using python is that I pretty never stumbled upon the equivalent of NullPointerException. Moreover, I wonder if that this convenience operator will do something more than hide bugs.

[Random832]

Xavier Combelle

<xavier....@gmail.com> writes:

> I'm curious on which API returning None, a major bonus on using python
> is that I pretty never stumbled upon the equivalent of
> NullPointerException.

It doesn't strictly have one; None is an object and you get the usual
TypeError, AttributeError, etc, upon using it in a place it's not expected.

_______________________________________________

[Guido van Rossum]

On Sat, Sep 19, 2015 at 2:39 PM, Random832 <rand...@fastmail.com> wrote:

Xavier Combelle
<xavier....@gmail.com> writes:

> I'm curious on which API returning None, a major bonus on using python
> is that I pretty never stumbled upon the equivalent of
> NullPointerException.

It doesn't strictly have one; None is an object and you get the usual
TypeError, AttributeError, etc, upon using it in a place it's not expected.

Most often AttributeError. It's pretty common in large Python systems.

[Andrew Barnert via Python-ideas]

The TypeErrors usually come from novices. There are many of StackOverflow questions asking why they can't add spam.get_text() + "\n" where they don't show you the implementation of get_text, or the exception they got, but you just know they forgot a return statement at the end and the exception was a TypeError about adding NoneType and str.

[Serhiy Storchaka]

On 19.09.15 07:21, Guido van Rossum wrote:
> I do, but at least the '?' is part of an operator, not part of the name
> (as it is in Ruby?).

What to do with the "in" operator?

[Random832]

Serhiy Storchaka <stor...@gmail.com>
writes:

> On 19.09.15 07:21, Guido van Rossum wrote:
>> I do, but at least the '?' is part of an operator, not part of the name
>> (as it is in Ruby?).
>
> What to do with the "in" operator?

This is one of those things where we've got to decide which side it
applies to. None can be in a list, but not a string. And nothing can be
in None.

[Serhiy Storchaka]

On 20.09.15 09:46, Random832 wrote:
> Serhiy Storchaka <stor...@gmail.com>
> writes:
>> On 19.09.15 07:21, Guido van Rossum wrote:
>>> I do, but at least the '?' is part of an operator, not part of the name
>>> (as it is in Ruby?).
>> What to do with the "in" operator?
> This is one of those things where we've got to decide which side it
> applies to. None can be in a list, but not a string. And nothing can be
> in None.

All operators are either identifiers ("in", "is", "not"), or
nonalphabetic. Not mixes. There is no the "in=" operator and I guess
shouldn't be "?in".

[Steven D'Aprano]

On Sun, Sep 20, 2015 at 09:10:32AM +0300, Serhiy Storchaka wrote:
> On 19.09.15 07:21, Guido van Rossum wrote:
> >I do, but at least the '?' is part of an operator, not part of the name
> >(as it is in Ruby?).
>
> What to do with the "in" operator?

Absolutely nothing.

I'm not convinced that we should generalise this beyond the three
original examples of attribute access, item lookup and function call. I
think that applying ? to arbitrary operators is a case of "YAGNI". Or
perhaps, "You Shouldn't Need It".

Mark's original motivating use-case strikes me as both common and
unexceptional. We might write:

# spam may be None, or some object
result = spam or spam.attr
# better, as it doesn't misbehave when spam is falsey
result = None if spam is None else spam.attr


and it seems reasonable to me to want a short-cut for that use-case. But
the generalisations to arbitrary operators suggested by Guido strike me
as too much, too far. As he says, going downhill, and quickly.

Consider these two hypotheticals:

spam ?+ eggs
# None if spam is None or eggs is None else spam + eggs

needle ?in haystack
# None if needle is None or haystack is None else needle in haystack

Briefer (more concise) is not necessarily better. At the point you have
*two* objects in the one term that both need to be checked for None,
that is in my opinion a code smell and we shouldn't provide a short-cut
disguising that.

Technically, x.y x[y] and x(y) aren't operators, but for the sake of
convenience I'll call them such. Even though these are binary operators,
the ? only shortcuts according to the x, not the y. So we can call
these ?. ?[] ?() operators "pseudo-unary" operators rather than binary
operators.

Are there any actual unary operators we might want to apply this
uptalk/shrug operator to? There are (if I remember correctly) only three
unary operators: + - and ~. I don't think there are any reasonable
use-cases for writing (say):

value = ?-x

that justifies making this short-cut available.

So as far as I am concerned, the following conditions should apply:

- the uptalk/shrug ? "operator" should not apply to actual binary
operators where both operands need to be checked for None-ness
(e.g. arithmetic operators, comparison operators)

- it should not apply to arithmetic unary operators + - and ~

- it might apply to pseudo-operators where only the lefthand
argument is checked for None-ness, that is, x.y x[y] and
x(y), written as x?.y x?[y] and x?(y).


If I had to choose between generalising this to all operators, or not
having it at all, I'd rather not have it at all. A little bit of uptalk
goes a long way, once we have ? appearing all over the place in all
sorts of expressions, I think it's too much.

--
Steve

[Bruce Leban]

On Sat, Sep 19, 2015 at 9:21 AM, Guido van Rossum <gu...@python.org> wrote:

I forgot to think about the scope of the uptalk operator (i.e. what is skipped when it finds a None). There are some clear cases (the actual implementation should avoid double evaluation of the tested expression, of course):

  a.b?.c.d[x, y](p, q) === None if a.b is None else a.b.c.d[x, y](p, q)

  a.b?[x, y].c.d(p, q) === None if a.b is None else a.b[x, y].c.d(p, q)

  a.b?(p, q).c.d[x, y] === None if a.b is None else a.b(p, q).c.d[x, y]

This makes sense to me.

 

But what about its effect on other operators in the same expression? I think this is reasonable:

  a?.b + c.d === None if a is None else a.b + c.d

This is a bit weird to me. Essentially ?. takes precedence over a following +. But would you also expect it to take precedence over a preceding one as well? That's inconsistent.

c.d + a?.b === None if a is None else c.d + a.b

or

c.d + a?.b === c.d + None if a is None else c.d + a.b

I think that ?. ?[] and ?() should affect other operators at the same precedence level only, i.e.,  each other and . [] and (). This seems the most logical to me. And I just looked up the C# documentation on MSDN and it does the same thing: https://msdn.microsoft.com/en-us/library/dn986595.aspx

It also shouldn't escape out of comma-separated lists, argument lists, etc.:

  (a?.b, x) === ((None if a is None else a.b), x)

  f(a?.b) === f((None if a is None else a.b))

Agree. It also should not escape grouping parenthesis even though that might not be useful. It would be very weird if a parenthesized expression did something other than evaluate the expression inside it, period.

(a?.b).c === None.c if a is None else (a.b).c === temp = a?.b; temp.c

(x or a?.b).c === (x or (None if a is none else a.b)).c

Yes, None.c is going to raise an exception. That's better than just getting None IMHO.

--- Bruce

Check out my new puzzle book: http://J.mp/ingToConclusions

Get it free here: http://J.mp/ingToConclusionsFree (available on iOS)

[Andrew Barnert via Python-ideas]

On Sep 20, 2015, at 00:31, Steven D'Aprano <st...@pearwood.info> wrote:
>
>> On Sun, Sep 20, 2015 at 09:10:32AM +0300, Serhiy Storchaka wrote:
>>> On 19.09.15 07:21, Guido van Rossum wrote:
>>> I do, but at least the '?' is part of an operator, not part of the name
>>> (as it is in Ruby?).
>>
>> What to do with the "in" operator?
>
> Absolutely nothing.
>
> I'm not convinced that we should generalise this beyond the three
> original examples of attribute access, item lookup and function call. I
> think that applying ? to arbitrary operators is a case of "YAGNI". Or
> perhaps, "You Shouldn't Need It".

I agree. Seeing how far you can generalize something and whether you can come up with a simple rule that makes all of your use cases follow naturally can be fun, but it isn't necessarily the best design.

Also, by not trying to generalize uptalk-combined operators (or uptalk as a postfix unary operator of its own, which I was earlier arguing for...), the question of how we deal with ?? or ?= (if we want them) can be "the same way every other language does", rather than seeing what follows from the general rule and then convincing ourselves that's what we wanted.

Also, I think trying to generalize to all operators is a false generalization, since the things we're generalizing from aren't actually operators (and not just syntactically--e.g., stylistically, they're never surrounded by spaces--which makes a pretty big difference in the readability impact of a character as heavy as "?") in the first place.

Personally, I think ?? is the second most obviously useful after ?. (there's a reason it's the one with the oldest and widest pedigree); we need ?() because Python, unlike C# and friends, unifies member and method access; ?[] doesn't seem as necessary but it's such an obvious parallel to ?() that I think people will expect it; ?= is potentially as confusing as it is helpful. So, my suggestion would be just the first four. And keeping them simple, and consistent with other languages, no trying to extend the protection to other operators/accesses, no extra short-circuiting, nothing. So:

spam ?? eggs === spam if spam is not None else eggs
spam?.eggs === spam.eggs if spam is not None else None
spam?(eggs) === spam(eggs) if spam is not None else None
spam?[eggs] === spam[eggs] if spam is not None else None

That's easy to define, easy to learn and remember, and pretty consistent with other languages. The one big difference is that what you write as "spam?.eggs(cheese)" in C# has to be "spam?.eggs?(cheese)" in Python, but I don't think that's a big problem. After all, in Python, spam.eggs is a first-class object, and one that's commonly passed or stored, so the obvious way to look at "spam.eggs(cheese)" is as explicitly chaining two separate things together (a __getattr__ with a descriptor __get__, and a __call__), so why shouldn't uptalking both operations be explicit?

[Chris Angelico]

On Sun, Sep 20, 2015 at 5:31 PM, Steven D'Aprano <st...@pearwood.info> wrote:
> Technically, x.y x[y] and x(y) aren't operators, but for the sake of
> convenience I'll call them such. Even though these are binary operators,
> the ? only shortcuts according to the x, not the y. So we can call
> these ?. ?[] ?() operators "pseudo-unary" operators rather than binary
> operators.

That's how all Python's short-circuiting works - based on the value of
what's on the left, decide whether or not to evaluate what's on the
right. (Well, nearly all - if/else evaluates the middle first, but
same difference.) This is another form of short-circuiting; "x[y]"
evaluates x, then if that's None, doesn't bother evaluating y because
it can't affect the result.

ChrisA

[Paul Moore]

On 20 September 2015 at 08:31, Steven D'Aprano <st...@pearwood.info> wrote:
> I'm not convinced that we should generalise this beyond the three
> original examples of attribute access, item lookup and function call. I
> think that applying ? to arbitrary operators is a case of "YAGNI". Or
> perhaps, "You Shouldn't Need It".

Agreed.

Does this need to be an operator? How about the following:

class Maybe:
def __getattr__(self, attr): return None
def __getitem__(self, idx): return None
def __call__(self, *args, **kw): return None

def maybe(obj):
return Maybe() if obj is None else obj

attr = maybe(obj).spam
elt = maybe(obj)[n]
result = maybe(callback)(args)

The Maybe class could be hidden, and the Maybe() object a singleton
(making my poor naming a non-issue :-)) and if it's felt sufficiently
useful, the maybe() function could be a builtin.

Usage of the result of maybe() outside of the above 3 contexts should
simply be "not supported" - don't worry about trying to stop people
doing weird things, just make it clear that the intent is only to
support the 3 given idiomatic usages.

Paul.

[Oleg Broytman]

On Sun, Sep 20, 2015 at 12:05:52PM +0100, Paul Moore <p.f....@gmail.com> wrote:
> On 20 September 2015 at 08:31, Steven D'Aprano <st...@pearwood.info> wrote:
> > I'm not convinced that we should generalise this beyond the three
> > original examples of attribute access, item lookup and function call. I
> > think that applying ? to arbitrary operators is a case of "YAGNI". Or
> > perhaps, "You Shouldn't Need It".
>
> Agreed.
>
> Does this need to be an operator? How about the following:
>
> class Maybe:
> def __getattr__(self, attr): return None
> def __getitem__(self, idx): return None
> def __call__(self, *args, **kw): return None
>
> def maybe(obj):
> return Maybe() if obj is None else obj
>
> attr = maybe(obj).spam
> elt = maybe(obj)[n]
> result = maybe(callback)(args)
>
> The Maybe class could be hidden, and the Maybe() object a singleton
> (making my poor naming a non-issue :-)) and if it's felt sufficiently
> useful, the maybe() function could be a builtin.
>
> Usage of the result of maybe() outside of the above 3 contexts should
> simply be "not supported" - don't worry about trying to stop people
> doing weird things, just make it clear that the intent is only to
> support the 3 given idiomatic usages.

PyMaybe - a Python implementation of the Maybe pattern. Seems to be
quite elaborated.

https://github.com/ekampf/pymaybe

> Paul.

Oleg.
--
Oleg Broytman http://phdru.name/ p...@phdru.name
Programmers don't die, they just GOSUB without RETURN.

[Andrew Barnert via Python-ideas]

On Sep 20, 2015, at 04:05, Paul Moore <p.f....@gmail.com> wrote:
>
> Does this need to be an operator? How about the following:
>
> class Maybe:
> def __getattr__(self, attr): return None
> def __getitem__(self, idx): return None
> def __call__(self, *args, **kw): return None
>
> def maybe(obj):
> return Maybe() if obj is None else obj
>
> attr = maybe(obj).spam
> elt = maybe(obj)[n]
> result = maybe(callback)(args)

But try this for calling a method on a possibly-null object:

result = maybe(maybe(spam).eggs)(cheese)

[David Mertz]

Paul Moore's idea is WAAYY better than the ugly ? pseudo-operator.  `maybe()` reads just like a regular function (because it is), and we don't need to go looking for Perl (nor Haskell) in some weird extra syntax that will confuse beginners.

--

Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.

[Guido van Rossum]

Actually if anything reminds me of Haskell it's a 'Maybe' type. :-(

But I do side with those who find '?' too ugly to consider.

--Guido van Rossum (python.org/~guido)

[Mark E. Haase]

On the day I started this thread, I wrote a Python module that does what maybe() does. I hadn't seen PyMaybe yet, and I couldn't think of any good names for my module's functions, so my module was disappointingly ugly.

PyMaybe is exactly what I *wish* I had written that day. For comparison, here's the code from my first post in this thread and it's maybe-ized version.

    response = json.dumps({

        'created': created?.isoformat(),

        'updated': updated?.isoformat(),

        ...

    })

    response = json.dumps({

        'created': maybe(created).isoformat(),

        'updated': maybe(updated).isoformat(),

        ...

    })

Pros:

1. No modification to Python grammar.

2. More readable: it's easy to overlook ? when skimming quickly, but "maybe()" is easy to spot.

3. More intuitive: the name "maybe" gives a hint at what it might do, whereas if you've never seen "?." you would need to google it. (Googling punctuation is obnoxious.)

Cons:

1. Doesn't short circuit: "maybe(family_name).upper().strip()" will fail if family_name is None.[1] You might try "maybe(maybe(family_name).upper()).strip()", but that is tricky to read and still isn't quite right: if family_name is not None, then it *should* be an error if "upper" is not an attribute of it. The 2-maybes form covers up that error.

I'm sure there will be differing opinions on whether this type of operation should short circuit. Some will say that we shouldn't be writing code that way: if you need to chain calls, then use some other syntax. But I think the example of upper case & strip is a good example of a perfectly reasonable thing to do. These kinds of operations are pretty common when you're interfacing with some external system or external data that has a concept of null (databases, JSON, YAML, argparse, any thin wrapper around C library, etc.).

This conversation has really focused on the null aware attribute access, but the easier and more defensible use case is the null coalesce operator, spelled "??" in C# and Dart. It's easy to find popular packages that use something like "retries = default if default is not None else cls.DEFAULT" to supply default instances.[2] Other packages do something like "retries = default or cls.DEFAULT"[3], which is worse because it easy to overlook the implicit coalescing of the left operand. In fact, the top hit for "python null coalesce" is StackOverflow, and the top-voted answer says to use "or".[4] (The answer goes on to explain the nuance of using "or" to coalesce, but how many developers read that far?)

In the interest of finding some common ground, I'd like to get some feedback on the coalesce operator. Maybe that conversation will yield some insight into the other "None aware" operators.

A) Is coalesce a useful feature? (And what are the use cases?)

B) If it is useful, is it important that it short circuits? (Put another way, could a function suffice?)

C) If it should be an operator, is "??" an ugly spelling?

    >>> retries = default ?? cls.DEFAULT

D) If it should be an operator, are any keywords more aesthetically pleasing? (I expect zero support for adding a new keyword.)

    >>> retries = default else cls.DEFAULT

    >>> retries = try default or cls.DEFAULT

    >>> retries = try default else cls.DEFAULT

    >>> retries = try default, cls.DEFAULT

    >>> retries = from default or cls.DEFAULT

    >>> retries = from default else cls.DEFAULT

    >>> retries = from default, cls.DEFAULT

My answers:

A) It's useful: supplying default instances for optional values is an obvious and common use case.

B) It should short circuit, because the patterns it replaces (using ternary operator or "or") also do.

C) It's too restrictive to cobble a new operator out of existing keywords; "??" isn't hard to read when it is separated by whitespace, as Pythonistas typically do between a binary operator and its operands.

D) I don't find any of these easier to read or write than "??".

[1] I say "should", but actually PyMaybe does something underhanded so that this expression does not fail: "maybe(foo).upper()" returns a "Nothing" instance, not "None". But Nothing has "def __repr__(self): return repr(None)". So if you try to print it out, you'll think you have a None instance, but it won't behave like one. If you try to JSON serialize it, you get a hideously confusing error: "TypeError: None is not JSON serializable". For those not familiar: the JSON encoder can definitely serialize None: it becomes a JSON "null". A standard implementation of maybe() should _not_ work this way.

[2] https://github.com/shazow/urllib3/blob/master/urllib3/util/retry.py#L148

[3] https://github.com/kennethreitz/requests/blob/46ff1a9a543cc4d33541aa64c94f50f0a698736e/requests/hooks.py#L25

[4] http://stackoverflow.com/a/4978745/122763

Mark E. Haase
202-815-0201

[Steven D'Aprano]

On Sun, Sep 20, 2015 at 07:38:18PM +1000, Chris Angelico wrote:
> On Sun, Sep 20, 2015 at 5:31 PM, Steven D'Aprano <st...@pearwood.info> wrote:
> > Technically, x.y x[y] and x(y) aren't operators, but for the sake of
> > convenience I'll call them such. Even though these are binary operators,
> > the ? only shortcuts according to the x, not the y. So we can call
> > these ?. ?[] ?() operators "pseudo-unary" operators rather than binary
> > operators.
>
> That's how all Python's short-circuiting works - based on the value of
> what's on the left, decide whether or not to evaluate what's on the
> right. (Well, nearly all - if/else evaluates the middle first, but
> same difference.) This is another form of short-circuiting; "x[y]"
> evaluates x, then if that's None, doesn't bother evaluating y because
> it can't affect the result.

I think you are mistaken about x[y]:

py> None[print("side effect")]
side effect
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not subscriptable

That's why x?[y] is a proposal.


--
Steve

[Chris Angelico]

On Mon, Sep 21, 2015 at 1:50 PM, Steven D'Aprano <st...@pearwood.info> wrote:
> On Sun, Sep 20, 2015 at 07:38:18PM +1000, Chris Angelico wrote:
>> On Sun, Sep 20, 2015 at 5:31 PM, Steven D'Aprano <st...@pearwood.info> wrote:
>> > Technically, x.y x[y] and x(y) aren't operators, but for the sake of
>> > convenience I'll call them such. Even though these are binary operators,
>> > the ? only shortcuts according to the x, not the y. So we can call
>> > these ?. ?[] ?() operators "pseudo-unary" operators rather than binary
>> > operators.
>>
>> That's how all Python's short-circuiting works - based on the value of
>> what's on the left, decide whether or not to evaluate what's on the
>> right. (Well, nearly all - if/else evaluates the middle first, but
>> same difference.) This is another form of short-circuiting; "x[y]"
>> evaluates x, then if that's None, doesn't bother evaluating y because
>> it can't affect the result.
>
> I think you are mistaken about x[y]:
>
> py> None[print("side effect")]
> side effect
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> TypeError: 'NoneType' object is not subscriptable
>
> That's why x?[y] is a proposal.

Oops, that was a typo in my statement. I meant "x?[y]" should behave
that way - once it's discovered that x is None, the evaluation of y
can't affect the result, and so it doesn't get evaluated (as per the
normal short-circuiting rules). Yes, x[y] has to evaluate both x and y
(after all, the value of y is passed to __getitem__). Sorry for the
confusion.

ChrisA

[Steven D'Aprano]

On Sun, Sep 20, 2015 at 12:05:52PM +0100, Paul Moore wrote:
> On 20 September 2015 at 08:31, Steven D'Aprano <st...@pearwood.info> wrote:
> > I'm not convinced that we should generalise this beyond the three
> > original examples of attribute access, item lookup and function call. I
> > think that applying ? to arbitrary operators is a case of "YAGNI". Or
> > perhaps, "You Shouldn't Need It".
>
> Agreed.
>
> Does this need to be an operator? How about the following:

Sadly, I think it does.

Guido has (I think) ruled out the Null object design pattern, which
makes me glad because I think it is horrid. But your Maybe class below
is a watered down, weak version that (in my opinion) isn't worth
bothering with. See below.

class Maybe:
def __getattr__(self, attr): return None
def __getitem__(self, idx): return None
def __call__(self, *args, **kw): return None

def maybe(obj):
return Maybe() if obj is None else obj

And in action:

py> maybe("spam").upper() # Works fine.
'SPAM'
py> maybe(None).upper()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>

TypeError: 'NoneType' object is not callable

It also fails for chained lookups:

maybe(obj).spam['id'].ham

will fail for the same reason. You could write this:

maybe(maybe(obj).upper)()
maybe(maybe(maybe(obj).spam)['id']).ham

but that's simply awful. Avoiding that problem is why the Null object
returns itself, but we've rightly ruled that out.

This is why I think that if this is worth doing, it has to be some sort
of short-circuiting operator or pseudo-operator:

expression ? .spam.eggs.cheese

can short-circuit the entire chain .spam.eggs.cheese, not just the first
component. Otherwise, I don't think it's worth doing.



--
Steve

[C Anthony Risinger]

On Sat, Sep 19, 2015 at 7:06 AM, Steven D'Aprano <st...@pearwood.info> wrote:

On Sat, Sep 19, 2015 at 03:17:07AM -0500, C Anthony Risinger wrote:

> I really liked this whole thread, and I largely still do -- I?think -- but
> I'm not sure I like how `?` suddenly prevents whole blocks of code from
> being evaluated. Anything within the (...) or [...] is now skipped (IIUC)
> just because a `?` was added, which seems like it could have side effects
> on the surrounding state, especially since I expect people will use it for
> squashing/silencing or as a convenient trick after the fact, possibly in
> code they did not originally write.

I don't think this is any different from other short-circuiting
operators, particularly `and` and the ternary `if` operator:

result = obj and obj.method(expression)

result = obj.method(expression) if obj else default

In both cases, `expression` is not evaluated if obj is falsey. That's
the whole point.

Sure, but those all have white space and I can read what's happening. The `?` could appear anywhere without break. I don't like that, but, opinion.

 

> If the original example included a `?` like so:
>
>     response = json.dumps?({

>         'created': created?.isoformat(),
>         'updated': updated?.isoformat(),
>         ...
>     })
>

> should "dumps" be None, the additional `?` (although though you can barely
> see it) prevents *everything else* from executing.

We're still discussing the syntax and semantics of this, so I could be
wrong, but my understanding of this is that the *first* question mark
prevents the expressions in the parens from being executed:

json.dumps?( ... )

evaluates as None if json.dumps is None, otherwise it evaluates the
arguments and calls the dumps object. In other words, rather like this:

_temp = json.dumps  # temporary value
if _temp is None:
    response = None
else:
    response = _temp({
        'created': None if created is None else created.isoformat(),
        'updated': None if updated is None else updated.isoformat(),
        ...
        })
del _temp


except the _temp name isn't actually used. The whole point is to avoid
evaluating an expression (attribute looking, index/key lookup, function
call) which will fail if the object is None, and if you're not going to
call the function, why evaluate the arguments to the function?

 

Yes that is how I understand it as well. I'm suggesting it's hard to see. I understand the concept as "None cancellation", because if the left is None, the right is cancelled. This lead me here:

* This is great, want to use all the time!

* First-level language support, shouldn't I use? Does feels useful/natural

* How can I make my APIs cancellation-friendly?

* I can write None-centric APIs, that often collapse to None

* Now maybe user code does stuff like `patient.visits?.september?.records` to get all records in September (if any, else None)

* Since both `?` points would *prefer* None, if the result is None, I now have to jump around looking for who done it

* If I don't have debugger ATM, I'm breaking it up a lot for good 'ol print(...), only way

* I don't think I like this any more :(

I especially don't like the idea of seeing it multiple times quickly, and the potential impact to debugging. The truth is I want to like this but I feel like it opens a can of worms (as seen by all the wild operators this proposal "naturally" suggests).

 

> Usually when I want to use this pattern, I find I just need to write things
> out more. The concept itself vaguely reminds me of PHP's use of `@` for
> squashing errors.

I had to look up PHP's @ and I must say I'm rather horrified. According
to the docs, all it does is suppress the error reporting, it does
nothing to prevent or recover from errors. There's not really an
equivalent in Python, but I suppose this is the closest:

# similar to PHP's $result = @(expression);
try:
    result = expression
except:
    result = None


This is nothing like this proposal. It doesn't suppress arbitrary
errors. It's more like a conditional:

# result = obj?(expression)
if obj is None:
    result = None
else:
    result = obj(expression)


If `expression` raises an exception, it will still be raised, but only
if it is actually evaluated, just like anything else protected by an
if...else or short-circuit operator.

I did say vaguely :) but it is extremely hideous I agree. The part that made me think of this is the would be desire for things to become None (so, or example, wanting to avoid throwing typed/informative exceptions if possible) so they'd then be more useful with `?`.

--

C Anthony

[Sven R. Kunze]

On 21.09.2015 06:17, C Anthony Risinger wrote:
> Yes that is how I understand it as well. I'm suggesting it's hard to
> see. I understand the concept as "None cancellation", because if the
> left is None, the right is cancelled. This lead me here:
>
> * This is great, want to use all the time!
> * First-level language support, shouldn't I use? Does feels useful/natural
> * How can I make my APIs cancellation-friendly?
> * I can write None-centric APIs, that often collapse to None
> * Now maybe user code does stuff like
> `patient.visits?.september?.records` to get all records in September
> (if any, else None)
> * Since both `?` points would *prefer* None, if the result is None, I
> now have to jump around looking for who done it
> * If I don't have debugger ATM, I'm breaking it up a lot for good 'ol
> print(...), only way
> * I don't think I like this any more :(
>
> I especially don't like the idea of seeing it multiple times quickly,
> and the potential impact to debugging. The truth is I want to like
> this but I feel like it opens a can of worms (as seen by all the wild
> operators this proposal "naturally" suggests).
>

It's interesting to see that everybody who ponders more than a minute
about it, really fast comes to the same conclusion.

Best,
Sven

[Sven R. Kunze]

On 21.09.2015 04:35, Mark E. Haase wrote:
> A) Is coalesce a useful feature? (And what are the use cases?)

I limit myself to materializing default arguments as in:

def a(b=None):
b = b or {}
...

Because its a well known theme (and issue) of the mutability of default
arguments of Python.

> B) If it is useful, is it important that it short circuits? (Put
> another way, could a function suffice?)
> C) If it should be an operator, is "??" an ugly spelling?
>
> >>> retries = default ?? cls.DEFAULT
>

The only difference between "or" and "??" is that "??" is None only,
right? At least to me, the given use case above does not justify the
introduction of "??".

> D) If it should be an operator, are any keywords more aesthetically
> pleasing? (I expect zero support for adding a new keyword.)
>
> >>> retries = default else cls.DEFAULT
> >>> retries = try default or cls.DEFAULT
> >>> retries = try default else cls.DEFAULT
> >>> retries = try default, cls.DEFAULT
> >>> retries = from default or cls.DEFAULT
> >>> retries = from default else cls.DEFAULT
> >>> retries = from default, cls.DEFAULT
>
>
> My answers:
>
> A) It's useful: supplying default instances for optional values is an
> obvious and common use case.

Yes, "or" suffices in that case.

> B) It should short circuit, because the patterns it replaces (using
> ternary operator or "or") also do.

They look ugly and unpleasant because they remind you to reduce the
usage of None; not to make dealing with it more pleasant.

> C) It's too restrictive to cobble a new operator out of existing
> keywords; "??" isn't hard to read when it is separated by whitespace,
> as Pythonistas typically do between a binary operator and its operands.
> D) I don't find any of these easier to read or write than "??".

"or" is easier to type (no special characters), I don't need to explain
it to new staff, and it's more pleasant to the eye.

I remember my missis telling me, after I showed her some C# code, that
programmers tend to like weird special characters. Well, that might
certainly be true. Special characters increase the visual noise and the
mental strain when reading. They make the lines they are in special. I
don't see anything special with "or" and with the single use case I have
for it. :)

Best,
Sven

[Chris Angelico]

On Mon, Sep 21, 2015 at 2:52 PM, Sven R. Kunze <srk...@mail.de> wrote:
> I limit myself to materializing default arguments as in:
>
> def a(b=None):
> b = b or {}
> ...

As long as you never need to pass in a specific empty dictionary,
that's fine. That's the trouble with using 'or' - it's not checking
for None, it's checking for falsiness.

ChrisA

[Sven R. Kunze]

On 21.09.2015 06:57, Chris Angelico wrote:
> On Mon, Sep 21, 2015 at 2:52 PM, Sven R. Kunze <srk...@mail.de> wrote:
>> I limit myself to materializing default arguments as in:
>>
>> def a(b=None):
>> b = b or {}
>> ...
> As long as you never need to pass in a specific empty dictionary,
> that's fine. That's the trouble with using 'or' - it's not checking
> for None, it's checking for falsiness.

True. Although I rarely pass a dynamic value to parameters with default
arguments. But you are right, so what does this mean for "??" ?

[Paul Moore]

On 20 September 2015 at 23:50, Guido van Rossum <gu...@python.org> wrote:
> Actually if anything reminds me of Haskell it's a 'Maybe' type. :-(

I warned you my choice of names was poor :-)
Paul

[Stephen J. Turnbull]

Mark E. Haase writes:

> This conversation has really focused on the null aware attribute access,
> but the easier and more defensible use case is the null coalesce operator,
> spelled "??" in C# and Dart. It's easy to find popular packages that use
> something like "retries = default if default is not None else cls.DEFAULT"

To me, it's less defensible. Eg, currently TOOWTDI for "??" is the
idiom quoted above. I sorta like the attribute access, attribute
fetch, and function call versions, though I probably won't use them.

Also some functions need to accept None as an actual argument, and the
module defines a module-specific sentinel. The inability to handle
such sentinels is a lack of generality that the "x if x is not
sentinel else y" idiom doesn't suffer from, so "??" itself can't
become TOOWTDI.

I don't write "def foo(default):" (ever that I can recall), so using
"default" in

retries = default if default is not None else cls.DEFAULT

confuses me. Realistically, I would be writing

retries = retries if retries is not None else cls.RETRIES

(or perhaps the RHS would be "self.retries"). That doesn't look that
bad to me (perhaps from frequent repetition). It's verbose, but I
don't see a need to chain it, unlike "?.". For "?.", some Pythonistas
would say "just don't", but I agree that often it's natural to chain.

> to supply default instances.[2] Other packages do something like
> "retries = default or cls.DEFAULT"[3], which is worse because it
> easy to overlook the implicit coalescing of the left operand.

Worse? It's true that it's more risky because it's all falsies, not
just the None sentinel, but I think "consenting adults" applies here.

I don't know about the packages you looked at, but I often use
"x = s or y" where I really want to trap the falsey value of the
expected type, perhaps as well as None, and I use the "x if s is not
sentinel else y" idiom to substitute default values. I also use "or"
in scripty applications and unit test setup functions where I want
compact expression and I don't expect long-lived objects to be passed
so I can easily figure out where the non-None falsey came from anyway.

> A) Is coalesce a useful feature? (And what are the use cases?)

Yes, for the whole group of operators. Several use cases for the
other operators have already been proposed, but I wouldn't use them
myself in current or past projects, and don't really foresee that
changing. -0 for the group on the IAGNI principle.

But for "??" specifically, it's just more compact AFAICS. I don't see
where I would use x ?? y ?? z, so the compactness doesn't seem like
that great a benefit. In practice, I think the use cases for "??"
would be a strict subset of the use cases for the ternary operator, so
you have to argue that "this special case *is* special enough" to have
its own way to do it. I don't think it is. -1

> C) If it should be an operator, is "??" an ugly spelling?
>
> >>> retries = default ?? cls.DEFAULT

Looks like metasyntax from pseudo-code that didn't get fixed to me.
That would probably change if other ?x operators were added though.

I have no comment on short-circuiting (no relevant experience), or
keyword vs. punctuation spellings. On second thought:

> D) If it should be an operator, are any keywords more aesthetically
> pleasing? (I expect zero support for adding a new keyword.)
>
> >>> retries = default else cls.DEFAULT

I kinda like this if-less else syntax for the symmetry with else-less
if. But on second thought I think it would persistently confuse me
when reading, because it would be extremely natural to expect it to be
another way of spelling "default or cls.DEFAULT". "try ... else ..."
also has its attraction, but I suppose that would fail for the same
reasons that the ternary operator is spelled "x if y else z" rather
than "if y then x else z".

[Andrew Barnert via Python-ideas]

On Sep 21, 2015, at 01:48, Stephen J. Turnbull <ste...@xemacs.org> wrote:

>>>>> retries = default else cls.DEFAULT
>
> I kinda like this if-less else syntax for the symmetry with else-less
> if.

How do you parse this:

a if b else c else d

Feel free to answer either as a human reader or as CPython's LL(1) parser.

[Paul Moore]

On 21 September 2015 at 03:35, Mark E. Haase <meh...@gmail.com> wrote:
> A) Is coalesce a useful feature? (And what are the use cases?)

There seem to be a few main use cases:

1. Dealing with functions that return a useful value or None to signal
"no value". I suspect the right answer here is actually to rewrite the
function to not do that in the first place. "Useful value or None"
seems like a reasonable example of an anti-pattern in Python.

2. People forgetting a return at the end of the function. In that
case, the error, while obscure, is reasonable, and should be fixed by
fixing the function, not by working around it in the caller.

3. Using a library (or other function outside your control) that uses
the "useful value or None" idiom. You have to make the best of a bad
job here, but writing an adapter function that hides the complexity
doesn't seem completely unreasonable. Nor does just putting the test
inline and accepting that you're dealing with a less than ideal API.

4. Any others? I can't think of anything.

Overall, I don't think coalesce is *that* useful, given that it seems
like it'd mainly be used in situations where I'd recommend a more
strategic fix to the code.

> B) If it is useful, is it important that it short circuits? (Put another
> way, could a function suffice?)

Short circuiting is important, but to me that simply implies that the
"useful value or None" approach is flawed *because* it needs
short-circuiting to manage. In lazy languages like Haskell, the Maybe
type is reasonable because short-circuiting is a natural consequence
of laziness, and so not a problem. In languages like C#, the use of
null as a sentinel probably goes back to C usage of NULL (i.e., it may
not be a good approach there either, but history and common practice
make it common enough that a fix is needed).

> C) If it should be an operator, is "??" an ugly spelling?
>
> >>> retries = default ?? cls.DEFAULT

Arbitrary punctuation as operators is not natural in Python, something
like this should be a keyword IMO.

> D) If it should be an operator, are any keywords more aesthetically
> pleasing? (I expect zero support for adding a new keyword.)
>
> >>> retries = default else cls.DEFAULT
> >>> retries = try default or cls.DEFAULT
> >>> retries = try default else cls.DEFAULT
> >>> retries = try default, cls.DEFAULT
> >>> retries = from default or cls.DEFAULT
> >>> retries = from default else cls.DEFAULT
> >>> retries = from default, cls.DEFAULT

Reusing existing keywords (specifically, all of the above) looks
clumsy and forced to me. I agree that proposals to add a new keyword
will probably never get off the ground, but none of the above
suggestions look reasonable to me, and I can't think of anything else
that does (particularly if you add "must be parseable" as a
restriction!)

Overall, I'm -0.5 on a "coalesce" operator. I can't see it having
sufficient value, and I can't think of a syntax I'd consider
justifying it. But if someone were to come up with a Guido-like
blindingly obvious way to spell the operation, I would be fine with
that (and may even start using it more often than I think).

Paul

[Chris Angelico]

On Mon, Sep 21, 2015 at 8:55 PM, Paul Moore <p.f....@gmail.com> wrote:
> There seem to be a few main use cases:
>
> 1. Dealing with functions that return a useful value or None to signal
> "no value". I suspect the right answer here is actually to rewrite the
> function to not do that in the first place. "Useful value or None"
> seems like a reasonable example of an anti-pattern in Python.

The alternative being to raise an exception? It's generally easier,
when you can know in advance what kind of object you're expecting, to
have a None return when there isn't one. For example, SQLAlchemy has
.get(id) to return the object for a given primary key value, and it
returns None if there's no such row in the database table - having to
wrap that with try/except would be a pain. This isn't an error
condition, and it's not like the special case of iteration (since an
iterator could yield any value, it's critical to have a non-value way
of signalling "end of iteration"). I don't want to see everything
forced to "return or raise" just because someone calls this an
anti-pattern.

ChrisA

[Paul Moore]

On 21 September 2015 at 14:27, Chris Angelico <ros...@gmail.com> wrote:
> On Mon, Sep 21, 2015 at 8:55 PM, Paul Moore <p.f....@gmail.com> wrote:
>> There seem to be a few main use cases:
>>
>> 1. Dealing with functions that return a useful value or None to signal
>> "no value". I suspect the right answer here is actually to rewrite the
>> function to not do that in the first place. "Useful value or None"
>> seems like a reasonable example of an anti-pattern in Python.
>
> The alternative being to raise an exception? It's generally easier,
> when you can know in advance what kind of object you're expecting, to
> have a None return when there isn't one. For example, SQLAlchemy has
> .get(id) to return the object for a given primary key value, and it
> returns None if there's no such row in the database table - having to
> wrap that with try/except would be a pain. This isn't an error
> condition, and it's not like the special case of iteration (since an
> iterator could yield any value, it's critical to have a non-value way
> of signalling "end of iteration"). I don't want to see everything
> forced to "return or raise" just because someone calls this an
> anti-pattern.

Agreed, that's not what should happen.

It's hard to give examples without going into specific cases, but as
an example, look at dict.get. The user can supply a "what to return if
the key doesn't exist" argument. OK, many people leave it returning
the default None, but they don't *have* to - dict.get itself doesn't
do "useful value or None", it does "useful value or user-supplied
default".

All I'm saying is that people should look at *why* their functions
return None instead of a useful result, and see if they can do better.
My contention is that (given free rein) many times they can. Of course
not all code has free rein, not all developers have the time to look
for perfect APIs, etc. But in that case, returning a placeholder None
(and accepting a little ugliness at the call site) isn't an impossible
price to pay.

Nothing more than "I don't think the benefit justifies adding a new
operator to Python".

Paul

[Steven D'Aprano]

On Mon, Sep 21, 2015 at 11:55:55AM +0100, Paul Moore wrote:
> On 21 September 2015 at 03:35, Mark E. Haase <meh...@gmail.com> wrote:
> > A) Is coalesce a useful feature? (And what are the use cases?)
>
> There seem to be a few main use cases:
>
> 1. Dealing with functions that return a useful value or None to signal
> "no value". I suspect the right answer here is actually to rewrite the
> function to not do that in the first place. "Useful value or None"
> seems like a reasonable example of an anti-pattern in Python.

I think that's a bit strong. Or perhaps much too strong.

There are times where you can avoid the "None or value" pattern, since
there is a perfectly decent empty value you can use instead of None.
E.g. if somebody doesn't have a name, you can use "" instead of None,
and avoid special treatment.

But that doesn't always work. Suppose you want an optional (say) Dog
object. There isn't such a thing as an empty Dog, so you have to use
some other value to represent the lack of Dog. One could, I suppose,
subclass Dog and build a (singleton? borg?) NoDog object, but that's
overkill and besides it doesn't scale well if you have multiple types
that need the same treatment.

So I don't think it is correct, or helpful, to say that we should avoid
the "None versus value" pattern. Sometimes we can naturally avoid it,
but it also has perfectly reasonable uses.

> Overall, I don't think coalesce is *that* useful, given that it seems
> like it'd mainly be used in situations where I'd recommend a more
> strategic fix to the code.

Go back to the original use-case given, which, paraphrasing, looks
something like this:

result = None if value is None else value['id'].method()

I don't think we can reject code like the above out of hand as
un-Pythonic or an anti-pattern. It's also very common, and a little
verbose. It's not bad when the value is a name, but sometimes it's an
expression, in which case it's both verbose and inefficient:

result = None if spam.eggs(cheese) is None else spam.eggs(cheese)['id'].method()

Contrast:

result = spam.eggs(cheese)?['id'].method()

which only calculates the expression to the left of the ?[ once.

An actual real-life example where we work around this by using a
temporary name that otherwise isn't actually used for anything:

mo = re.match(needle, haystack)
if mo:
substr = mo.group()
else:
substr = None


I think it is perfectly reasonable to ask for syntactic sugar to avoid
having to write code like the above:

substr = re.match(needle, haystack)?.group()


That's not to say we necessarily should add sugar for this, since
there is no doubt there are disadvantages as well (mostly that many
people dislike the ? syntax), but in principle at least it would
certainly be nice to have and useful.

> > B) If it is useful, is it important that it short circuits? (Put another
> > way, could a function suffice?)
>
> Short circuiting is important, but to me that simply implies that the
> "useful value or None" approach is flawed *because* it needs
> short-circuiting to manage.

Nothing needs short-circuiting, at least in a language with imperative
assignment statements. You can always avoid the need for short-circuits
with temporary variables, and sometimes that's the right answer: not
everything needs to be a one-liner, or an expression.

But sometimes it is better if it could be.

> > C) If it should be an operator, is "??" an ugly spelling?
> >
> > >>> retries = default ?? cls.DEFAULT

I assume the ?? operator is meant as sugar for:

retries = cls.DEFAULT if default is None else default

I prefer to skip the "default" variable and use the standard idiom:

if retries is None:
retries = cls.DEFAULT

I also worry about confusion caused by the asymmetry between ?? and the
other three ? cases:

# if the left side is None, return None, else evaluate the right side
spam?.attr
spam?['id']
spam?(arg)

# if the left side is None, return the right side, else return the left
spam ?? eggs

but perhaps I'm worried over nothing.


--
Steve

[Paul Moore]

On 21 September 2015 at 15:41, Steven D'Aprano <st...@pearwood.info> wrote:
> An actual real-life example where we work around this by using a
> temporary name that otherwise isn't actually used for anything:
>
> mo = re.match(needle, haystack)
> if mo:
> substr = mo.group()
> else:
> substr = None
>
>
> I think it is perfectly reasonable to ask for syntactic sugar to avoid
> having to write code like the above:
>
> substr = re.match(needle, haystack)?.group()

Well, (1) Mark had focused on the "coalesce" operator ??, not the ?.
variant, and that is less obviously useful here, and (2) I find the
former version more readable. YMMV on readability of course - which is
why I added the proviso that if someone comes up with an "obviously
right" syntax, I may well change my mind. But the options suggested so
far are all far less readable than a simple multi-line if (maybe with
a temporary variable) to me, at least.

By the way, in your example you're passing on the "none or useful"
property by making substr be either the matched value or None. In real
life, I'd probably do something more like

mo = re.match(needle, haystack)
if mo:

process(mo.group())
else:
no_needle()

possibly with inline code if process or no_needle were simple. But it
is of course easy to pick apart examples - real code isn't always that
tractable. For example we get (what seems to me like) a *lot* of bug
reports about "None has not attribute foo" style errors in pip. My
comments here are based on my inclinations about how I would fix them
in pip - I'd always go back to *why* we got a None, and try to avoid
getting the None in the first place. But that's not always easy to do.
Again, of course, YMMV.

Paul

[Ron Adam]

On 09/20/2015 11:57 PM, Chris Angelico wrote:
> On Mon, Sep 21, 2015 at 2:52 PM, Sven R. Kunze<srk...@mail.de> wrote:
>> >I limit myself to materializing default arguments as in:
>> >
>> >def a(b=None):
>> > b = b or {}
>> > ...
> As long as you never need to pass in a specific empty dictionary,
> that's fine. That's the trouble with using 'or' - it's not checking

> for None, it's checking for falseness.

From reading these, I think the lowest-level/purest change would be to
accommodate testing for "not None". Something I've always thought
Python should be able to do in a nicer more direct way.

We could add a "not None" specific boolean operators just by appending !
to them.

while! x: <--> while x != None:
if! x: <--> if x != None:

a or! b <--> b if a != None else a
a and! b <--> a if a != None else b
not! x <--> x if x != None else None

Those expressions on the right are very common and are needed because of
None, False, and 0, are all False values.

It would make for much simpler expressions and statements where they are
used and be more efficient as these are likely to be in loops going over
*many* objects. So it may also result in a fairly nice speed
improvement for many routines.

While the consistency argument says "if!" should be equivalent to "if
not", I feel the practicality argument leans towards it being specific
to "if obj != None".

I believe testing for "not None" is a lot more common than testing for
"None". Usually the only difference is how the code is arranged.

I like how it simplifies/clarifies the common cases above. It would be
especially nice in comprehensions.

Cheers,
Ron

[Guido van Rossum]

Just to cut this thread short, I'm going to reject PEP 505, because ? is just too ugly to add to Python IMO. Sorry.

I commend Mark for his clean write-up, without being distracted, giving some good use cases. I also like that he focused on a minimal addition to the language and didn't get distracted by hyper-generalizations.

I also like that he left out f?(...) -- the use case is much weaker; usually it's the object whose method you're calling that might be None, as in title?.upper().

Some nits for the PEP:

- I don't think it ever gives the priority for the ?? operator. What would "a ?? b or c" mean?

- You don't explain why it's x ?? y but x ?= y. I would have expected either x ? y or x ??= y.

- You don't explain or show how far ?. reaches; I assume x?y.z is equivalent to None if x is None else x.y.z, so you don't have to write x?.y?.z just to handle x.y.z if x is None.

- The specification section is empty.

[Mark E. Haase]

PEP-505 isn't anywhere close to being finished. I only submitted the draft because somebody off list asked me to send a draft so I could get a PEP number assigned. So I literally sent him what I had open in my text editor, which was just a few minutes of brain dumping and had several mistakes (grammatical and technical).

If there's absolutely no point in continuing to work on it, I'll drop it. But from the outset, I thought the plan was to present this in its best light (and similar to the ternary operator PEP, offer several alternatives) if for no other reason than to have a good record of the reasoning for rejecting it.

I'm sorry if I misunderstood the PEP process; I would have kept it to myself longer if I knew the first submission was going to be reviewed critically. I thought this e-mail chain was more of an open discussion on the general idea, not specifically a referendum on the PEP itself.

[Sven R. Kunze]

On 21.09.2015 11:05, Andrew Barnert via Python-ideas wrote:
> On Sep 21, 2015, at 01:48, Stephen J. Turnbull <ste...@xemacs.org> wrote:
>
>>>>>> retries = default else cls.DEFAULT
>> I kinda like this if-less else syntax for the symmetry with else-less
>> if.

That's cool. It reads nice (at least for a non-native speaker). Also
chaining else reads nice:

final_value = users_value else apps_value else systems_value

> How do you parse this:
>
> a if b else c else d
>
> Feel free to answer either as a human reader or as CPython's LL(1) parser.

Use parentheses if you mix up if-else and else. ;)

Btw. the same applies for: a + b * c + d
If you don't know from you education that b*c would have been evaluated
first, then it's not obvious either.

Best,
Sven

[Sven R. Kunze]

On 21.09.2015 15:27, Chris Angelico wrote:
> On Mon, Sep 21, 2015 at 8:55 PM, Paul Moore <p.f....@gmail.com> wrote:
>> There seem to be a few main use cases:
>>
>> 1. Dealing with functions that return a useful value or None to signal
>> "no value". I suspect the right answer here is actually to rewrite the
>> function to not do that in the first place. "Useful value or None"
>> seems like a reasonable example of an anti-pattern in Python.
> The alternative being to raise an exception? It's generally easier,
> when you can know in advance what kind of object you're expecting, to
> have a None return when there isn't one. For example, SQLAlchemy has
> .get(id) to return the object for a given primary key value, and it
> returns None if there's no such row in the database table - having to
> wrap that with try/except would be a pain. This isn't an error
> condition, and it's not like the special case of iteration (since an
> iterator could yield any value, it's critical to have a non-value way
> of signalling "end of iteration"). I don't want to see everything
> forced to "return or raise" just because someone calls this an
> anti-pattern.

I don't think both approaches are mutual exclusive. They can both exist
and provide whenever I need the right thing.

Depending on the use-case, one needs to decide:

If I know, the value definitely needs to be a dictionary, I use dict[...].
If I know, the value is definitely optional and I can't do anything
about it, I use dict.get('key'[, default]).
If I definitely don't know, I use dict[...] to get my hands on a real
example with out that key if that every happens and don't waste time for
special-handling a possible None return value.

Best,
Sven

[Stephen J. Turnbull]

Andrew Barnert writes:
> On Sep 21, 2015, at 01:48, Stephen J. Turnbull <ste...@xemacs.org> wrote:
>
> >>>>> retries = default else cls.DEFAULT
> >
> > I kinda like this if-less else syntax for the symmetry with else-less
> > if.
>
> How do you parse this:
>
> a if b else c else d
>
> Feel free to answer either as a human reader or as CPython's LL(1)
> parser.

I don't know what an LL(1) parser could do offhand. As a human, I
would parse that greedily as (a if b else c) else d.

But the point's actually moot, as I'm -1 on the "??" operator in any
form in favor of the explicit "a if a is not None else b" existing
syntax. And to be honest, the fact that a truly symmetric "if-less
else" would have "or" semantics, not "??" semantics, bothers me more
than the technical issue of whether anybody could actually parse it.

[Sven R. Kunze]

On 21.09.2015 18:27, Sven R. Kunze wrote:

If I know, the value definitely needs to be IN the dictionary, I use dict[...].

typo

[Random832]

On Mon, Sep 21, 2015, at 12:29, Stephen J. Turnbull wrote:
> I don't know what an LL(1) parser could do offhand. As a human, I
> would parse that greedily as (a if b else c) else d.

That's not greedy. The greedy parsing is (a if b else (c else d)).

[Guido van Rossum]

On Mon, Sep 21, 2015 at 8:58 AM, Mark E. Haase <meh...@gmail.com> wrote:

PEP-505 isn't anywhere close to being finished. I only submitted the draft because somebody off list asked me to send a draft so I could get a PEP number assigned. So I literally sent him what I had open in my text editor, which was just a few minutes of brain dumping and had several mistakes (grammatical and technical).

If there's absolutely no point in continuing to work on it, I'll drop it. But from the outset, I thought the plan was to present this in its best light (and similar to the ternary operator PEP, offer several alternatives) if for no other reason than to have a good record of the reasoning for rejecting it.

I'm sorry if I misunderstood the PEP process; I would have kept it to myself longer if I knew the first submission was going to be reviewed critically. I thought this e-mail chain was more of an open discussion on the general idea, not specifically a referendum on the PEP itself.

I apologize for having misunderstood the status of your PEP. I think it would be great if you finished the PEP. As you know the ? operator has its share of fans as well as detractors, and I will happily wait until more of a consensus appears. I hope you can also add a discussion to the PEP of ideas (like some of the hyper-generalizations) that were considered and rejected -- summarizing a discussion is often a very important goal of a PEP. I think you have made a great start already!

--Guido

 

On Mon, Sep 21, 2015 at 11:40 AM, Guido van Rossum <gu...@python.org> wrote:

Just to cut this thread short, I'm going to reject PEP 505, because ? is just too ugly to add to Python IMO. Sorry.

I commend Mark for his clean write-up, without being distracted, giving some good use cases. I also like that he focused on a minimal addition to the language and didn't get distracted by hyper-generalizations.

I also like that he left out f?(...) -- the use case is much weaker; usually it's the object whose method you're calling that might be None, as in title?.upper().

Some nits for the PEP:

- I don't think it ever gives the priority for the ?? operator. What would "a ?? b or c" mean?

- You don't explain why it's x ?? y but x ?= y. I would have expected either x ? y or x ??= y.

- You don't explain or show how far ?. reaches; I assume x?y.z is equivalent to None if x is None else x.y.z, so you don't have to write x?.y?.z just to handle x.y.z if x is None.

- The specification section is empty.

--

--Guido van Rossum (python.org/~guido)

--

Mark E. Haase
202-815-0201

[Ryan Gonzalez]

What about re-using try? Crystal does this (http://play.crystal-lang.org/#/r/gf5):

v = "ABC"

puts nil == v.try &.downcase # prints true

v = nil

puts nil == v.try &.downcase # prints false

Python could use something like:

v = 'ABC'

print(v try.downcase is None) # prints False

v = None

print(v try.downcase is None) # prints True

(Of course, the syntax would be a little less...weird!)

_______________________________________________
Python-ideas mailing list
Python...@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

--

Ryan

[ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong.

http://kirbyfan64.github.io/

 

[MRAB]

On 2015-09-21 17:29, Stephen J. Turnbull wrote:
> Andrew Barnert writes:
> > On Sep 21, 2015, at 01:48, Stephen J. Turnbull <ste...@xemacs.org> wrote:
> >
> > >>>>> retries = default else cls.DEFAULT
> > >
> > > I kinda like this if-less else syntax for the symmetry with else-less
> > > if.
> >
> > How do you parse this:
> >
> > a if b else c else d
> >
> > Feel free to answer either as a human reader or as CPython's LL(1)
> > parser.
>
> I don't know what an LL(1) parser could do offhand. As a human, I
> would parse that greedily as (a if b else c) else d.
>

'else' is being used like 'or', except when it belongs to 'if'.

I can't see a way of handling that.

It would result in a syntax error.

[Terry Reedy]

On 9/21/2015 10:56 AM, Paul Moore wrote:

> By the way, in your example you're passing on the "none or useful"
> property by making substr be either the matched value or None.

I agree that dealing with None immediately is better.

In real
> life, I'd probably do something more like
>
> mo = re.match(needle, haystack)
> if mo:
> process(mo.group())
> else:
> no_needle()

try:
process(re.match(needle, haystack).group())
except AttributeError: # no match
no_needle()

is equivalent unless process can also raise AttributeError.


--
Terry Jan Reedy

[Terry Reedy]

On 9/21/2015 11:28 AM, Ron Adam wrote:

> We could add a "not None" specific boolean operators just by appending !
> to them.
>
> while! x: <--> while x != None:
> if! x: <--> if x != None:
>
> a or! b <--> b if a != None else a
> a and! b <--> a if a != None else b
> not! x <--> x if x != None else None

'!= None' should be 'is not None' in all examples. Since 'is not None'
is a property of the object, so I think any abbreviation should be
applied to the object, not the operator. "while x!", etcetera

--
Terry Jan Reedy

[Terry Reedy]

On 9/21/2015 1:07 PM, Guido van Rossum wrote:

> I apologize for having misunderstood the status of your PEP. I think it
> would be great if you finished the PEP. As you know the ? operator has
> its share of fans as well as detractors, and I will happily wait until
> more of a consensus appears.

Add me to the detractors of what I have read so far ;-).

In arithmetic, 1/0 and 0/0 both stop the calculation. My hand
calculator literally freezes until I hit 'on' or 'all clear'. Early
computers also stopped, maybe with an instruction address and core dump.
Three orthogonal solutions are: test y before x/y, so one can do
something else; introduce catchable exceptions, so one can do something
else; introduce contagious special objects ('inf' and 'nan'), which at
some point can be tested for, so one can do something else. Python
introduced 'inf' and 'nan' but did not use them to replace
ZeroDivisionError.

Some languages lacking exceptions introduce a contagious null object.
Call it Bottom. Any operation on Bottom yields Bottom. Python is not
such a language. None is anti-contagious; most operations raise an
exception.

I agree with Paul Moore that propagating None is generally a bad idea.
It merely avoids the inevitable exception. Or is it inevitable? Trying
to avoid exceptions naturally leads to the hypergeneralization of
allowing '?' everywhere.

Instead of trying to turn None into Bottom, I think a better solution
would be a new, contagious, singleton Bottom object with every possible
special method, all returning Bottom. Anyone could write such for their
one use. Someone could put it on pypi to see if there how useful it
would be.

I agree with Ron Adam that the narrow issue is that bool(x) is False is
sometimes too broad and people dislike of spelling out 'x is not None'.
So abbreviate that with a unary operator; 'is not None', is a property
of objects, not operators. I think 'x!' or 'x?', either meaning 'x is
not None', might be better than a new binary operator. The former, x!,
re-uses ! in something close to its normal meaning: x really exists.

--
Terry Jan Reedy

[Guido van Rossum]

On Mon, Sep 21, 2015 at 2:23 PM, Terry Reedy <tjr...@udel.edu> wrote:

Add me to the detractors of what I have read so far ;-).

In arithmetic, 1/0 and 0/0 both stop the calculation.  My hand calculator literally freezes until I hit 'on' or 'all clear'.  Early computers also stopped, maybe with an instruction address and core dump.  Three orthogonal solutions are: test y before x/y, so one can do something else; introduce catchable exceptions, so one can do something else; introduce contagious special objects ('inf' and 'nan'), which at some point can be tested for, so one can do something else.  Python introduced 'inf' and 'nan' but did not use them to replace ZeroDivisionError.

Some languages lacking exceptions introduce a contagious null object. Call it Bottom.  Any operation on Bottom yields Bottom.  Python is not such a language. None is anti-contagious; most operations raise an exception.

I agree with Paul Moore that propagating None is generally a bad idea. It merely avoids the inevitable exception.  Or is it inevitable? Trying to avoid exceptions naturally leads to the hypergeneralization of allowing '?' everywhere.

Instead of trying to turn None into Bottom, I think a better solution would be a new, contagious, singleton Bottom object with every possible special method, all returning Bottom. Anyone could write such for their one use.  Someone could put it on pypi to see if there how useful it would be.

I think this is the PyMaybe solution. What I don't like about it is that it is dynamic -- when used incorrectly (or even correctly?) Bottom could end up being passed into code that doesn't expect it. That's bad -- "if x is None" returns False when x is Bottom, so code that isn't prepared for Bottom may well misbehave. In contrast, PEP 505 only affects code that is lexically near the ? operator.

(You may see a trend here. PEP 498 is also carefully designed to be locally-scoped.)

 

I agree with Ron Adam that the narrow issue is that bool(x) is False is sometimes too broad and people dislike of spelling out 'x is not None'. So abbreviate that with a unary operator; 'is not None', is a property of objects, not operators. I think 'x!' or 'x?', either meaning 'x is not None', might be better than a new binary operator. The former, x!, re-uses ! in something close to its normal meaning: x really exists.

I don't think the big issue is bool(x) being too broad. That's what the binary ?? operator is trying to fix, but to me the more useful operators are x?.y and x?[y], both of which would still require repetition of the part on the left when spelled using ??.

This is important when x is a more complex expression that is either expensive or has a side-effect. E.g. d.get(key)?.upper() would currently have to be spelled as (some variant of) "None if d.get(key) is None else d.get(key).upper()" and the ?? operator doesn't really help for the repetition -- it would still be "d.get(key) ?? d.get(key).upper()".

In general to avoid this repetition you have to introduce a local variable, but that's often awkward and interrupts the programmer's "flow". The ? solves that nicely. The key issue with this proposal to me is how it affects readability of code that uses it, given that there isn't much uniformity across languages in what ? means -- it could be part of a method name indicating a Boolean return value (Ruby) or a conditional operator (C and most of its descendents) or some kind of shortcut.

So this is the issue I have to deal with (and thought I had dealt with by prematurely rejecting the PEP, but I've had a change of heart and am now waiting for the PEP to be finished).

[Terry Reedy]

On 9/21/2015 5:48 PM, Guido van Rossum wrote:
> On Mon, Sep 21, 2015 at 2:23 PM, Terry Reedy
> <tjr...@udel.edu

> <mailto:tjr...@udel.edu>> wrote:

> I agree with Paul Moore that propagating None is generally a bad
> idea. It merely avoids the inevitable exception.

To me, this is the key idea in opposition to proposals that make
propagating None easier.

> I don't think the big issue is bool(x) being too broad. That's what the
> binary ?? operator is trying to fix, but to me the more useful operators
> are x?.y and x?[y], both of which would still require repetition of the
> part on the left when spelled using ??.
>
> This is important when x is a more complex expression that is either
> expensive or has a side-effect. E.g. d.get(key)?.upper() would currently
> have to be spelled as (some variant of)
> "None if d.get(key) is None else d.get(key).upper()"
> and the ?? operator doesn't really help for the
> repetition -- it would still be "d.get(key) ?? d.get(key).upper()".
>
> In general to avoid this repetition you have to introduce a local
> variable, but that's often awkward and interrupts the programmer's
> "flow".

try:
x = d.get(key).upper()
except AttributeError:
x = None

is also a no-repeat equivalent when d.values are all strings. I agree
than "x = d.get(key)?.upper()" is a plausible abbreviation. But I am
much more likely to want "x = ''" or another exception as the
alternative. I guess some other pythonistas like keeping None around
more than I do ;-).

[Guido van Rossum]

On Mon, Sep 21, 2015 at 3:45 PM, Terry Reedy <tjr...@udel.edu> wrote:

On 9/21/2015 5:48 PM, Guido van Rossum wrote:

On Mon, Sep 21, 2015 at 2:23 PM, Terry Reedy
<tjr...@udel.edu
<mailto:tjr...@udel.edu>> wrote:

    I agree with Paul Moore that propagating None is generally a bad
    idea. It merely avoids the inevitable exception.

To me, this is the key idea in opposition to proposals that make propagating None easier.

(I didn't write that, you [Terry] did. It looks like our mailers don't understand each other's quoting conventions. :-( )

 

I don't think the big issue is bool(x) being too broad. That's what the
binary ?? operator is trying to fix, but to me the more useful operators
are x?.y and x?[y], both of which would still require repetition of the
part on the left when spelled using ??.

This is important when x is a more complex expression that is either
expensive or has a side-effect. E.g. d.get(key)?.upper() would currently
have to be spelled as (some variant of)

     > "None if d.get(key) is None else d.get(key).upper()"
     > and the ?? operator doesn't really help for the

repetition -- it would still be "d.get(key) ?? d.get(key).upper()".

In general to avoid this repetition you have to introduce a local
variable, but that's often awkward and interrupts the programmer's
"flow".

try:
    x = d.get(key).upper()
except AttributeError:
    x = None

is also a no-repeat equivalent when d.values are all strings.  I agree than "x = d.get(key)?.upper()" is a plausible abbreviation.  But I am much more likely to want "x = ''" or another exception as the alternative.  I guess some other pythonistas like keeping None around more than I do ;-).

Eew. That try/except is not only very distracting and interrupts the flow of both the writer and the reader, it may also catch errors, e.g. what if the method being called raises an exception (not a problem with upper(), but definitely with user-defined methods).

[Carl Meyer]

On 09/21/2015 04:45 PM, Terry Reedy wrote:
> On 9/21/2015 5:48 PM, Guido van Rossum wrote:
>> On Mon, Sep 21, 2015 at 2:23 PM, Terry Reedy
>> <tjr...@udel.edu
>> <mailto:tjr...@udel.edu>> wrote:
>
>> I agree with Paul Moore that propagating None is generally a bad
>> idea. It merely avoids the inevitable exception.
>
> To me, this is the key idea in opposition to proposals that make
> propagating None easier.

[...]

> I guess some other pythonistas like keeping None around
> more than I do ;-).

I think it's one of those things that depends on what you're doing. From
a web-development perspective, you rarely keep _anything_ around for
very long, so there's rarely an issue of `None` sneaking in somewhere
unexpectedly and then causing a surprise exception way down the line.
Typical use cases are things like: "If this database query returns a
User, I want to get their name and return that in the JSON dict from my
API, otherwise I want None, which will be serialized to a JSON null,
clearly indicating that there is no user here."

My jaw dropped a bit when I saw it asserted in this thread that
functions returning "useful value or None" is an anti-pattern. I write
functions like that all the time, and I consider it a useful and
necessary Python idiom. I would hate to rewrite all that code to either
deal with exceptions or add default-value-argument boilerplate to all of
them; when "no result" is an expected and normal possibility from a
function, letting the calling code deal with None however it chooses is
much nicer than either of those options.

I don't love the ? syntax, but I would certainly use the feature
discussed here happily and frequently.

Carl

[Random832]

On Mon, Sep 21, 2015, at 17:48, Guido van Rossum wrote:
> This is important when x is a more complex expression that is either
> expensive or has a side-effect. E.g. d.get(key)?.upper() would currently
> have to be spelled as (some variant of) "None if d.get(key) is None else
> d.get(key).upper()" and the ?? operator doesn't really help for the
> repetition -- it would still be "d.get(key) ?? d.get(key).upper()".

?? is meant to use the right if the left *is* null, as I understand it.
So this isn't a problem it solves at all.

[Guido van Rossum]

On Mon, Sep 21, 2015 at 4:47 PM, Random832 <rand...@fastmail.com> wrote:

On Mon, Sep 21, 2015, at 17:48, Guido van Rossum wrote:
> This is important when x is a more complex expression that is either
> expensive or has a side-effect. E.g. d.get(key)?.upper() would currently
> have to be spelled as (some variant of) "None if d.get(key) is None else
> d.get(key).upper()" and the ?? operator doesn't really help for the
> repetition -- it would still be "d.get(key) ?? d.get(key).upper()".

?? is meant to use the right if the left *is* null, as I understand it.
So this isn't a problem it solves at all.

Sorry, my bad. Indeed, x ?? y tries to fix the issue that "x or y" uses y if x is falsey. Still this seems a lesser problem to me than the problem solved by x?.a and x?[y]. Most of the time in my code it is actually fine to use the default if the LHS is an empty string.

[Bruce Leban]

On Mon, Sep 21, 2015 at 3:56 PM, Carl Meyer <ca...@oddbird.net> wrote:

My jaw dropped a bit when I saw it asserted in this thread that
functions returning "useful value or None" is an anti-pattern. I write
functions like that all the time, and I consider it a useful and
necessary Python idiom. I would hate to rewrite all that code to either
deal with exceptions or add default-value-argument boilerplate to all of
them; when "no result" is an expected and normal possibility from a
function, letting the calling code deal with None however it chooses is
much nicer than either of those options.

+1

Some language features are "prescriptive," designed to encourage particular ways of writing things. Others are "respective," recognizing the variety of ways people write things and respecting that variety. Python has None and generally respects use of it. To say that using None is an anti-pattern is something I would strongly disagree with. Yes, NPE errors are a problem, but eliminating null/None does not eliminate those errors. It merely replaces one common error with an assortment of other errors.

I like the feature. I have been asking for features like this for years and the number of times I have written the longer forms is too many to count.

I like the ?.  ?[]  ?()  ?? syntax. I think:

(1) it's strongly related to the . [] () syntax;

(2) any syntax that uses a keyword is either not syntactically related to . [] () or mixes a keyword and punctuation, both of which I dislike;

(3) it's the same syntax as used in other languages (yes, Python is not C# or Dart but there's a good reason Python uses ^ for xor, ** for power, += for add to, etc.)

--- Bruce

[Matthias Bussonnier]

On Mon, Sep 21, 2015 at 2:48 PM, Guido van Rossum <gu...@python.org> wrote:

>
> In general to avoid this repetition you have to introduce a local variable,
> but that's often awkward and interrupts the programmer's "flow". The ?
> solves that nicely. The key issue with this proposal to me is how it affects
> readability of code that uses it, given that there isn't much uniformity
> across languages in what ? means -- it could be part of a method name
> indicating a Boolean return value (Ruby) or a conditional operator (C and
> most of its descendents) or some kind of shortcut.
>
> So this is the issue I have to deal with (and thought I had dealt with by
> prematurely rejecting the PEP, but I've had a change of heart and am now
> waiting for the PEP to be finished).
>

As we are in the process of writing a PEP and uses of ?/?? in other languages,
why not speak about thecurrent usage of `?` / `??` in the Python community ?

(I'll try to state only facts, excuse any passage that might seem like
a personal opinion)

Can the PEP include the fact that `? and `??` have been in use in the Scientific
Python community for 10 to 14 years now, and that any Scientific Python user who
have touched IPython will tell you that ? and ?? are for getting help.
(?? try to pull the source, while ? does not, but let's not get into details).

This include the fact that any IDE (like spyder) which use IPython under
the hood have this feature.

The usage of `?` is even visible on Python.org main page [3] (imgur screenshot),
which invite the user to launch an interactive console saying:

> object? -> Details about 'object', use 'object??' for extra details.
> In [1]:

leading the casual user thinking that this is a Python feature.

This fact is even including in Books and introduction to python. Sometime
without mentioning that the feature is IPython Specific, and does not work in
Python repl/scripts.

Examples in Cyrile's rossant "Learning IPython for Interactive
Computing and Data Visualization"[1]
introduce Python with the second code/repl example beeing about `?`.

Book extract :

> Some of these commands let you get some help or information about any
> Python function or object. For instance, have you ever had a doubt about how
> to use the super function to access parent methods in a derived class? Just type
> `super?` and you’ll find out. Appending `?` to any command or variable gives you all
> the information you need about it.

> In [1]: super?
> Type: type
> String Form:<type 'super'>
> Namespace: Python builtin
> ...

A google search also give for eaxample:
Python for beginners online tutorial[2] which does rapidly the same:

Tuto snippet:
> The "?" is very useful. If you type in `?` after a `len?`, you will see the
> documentation about the function len.
> Typing `?` after a name will give you information about the object attached to that name.

Doing even worse as they replace the IPython prompt `In[x]:` with
`>>>` literally showing
that `>>> len?` works. Which imply that it should work on a plain Python REPL.

As someone that have to regularly teach Python, and interact with new
Python users,
it will be hard to explain that `?` and `??` have different meaning
depending on the context,
and that most book on Scientific Python are wrong/inaccurate.

From the current state of the PEP/proposal I'm guessing we should be
able to distinguish
Null Coalescing operation (or whatever name you want to give them) in
Python 3.6+
from actual help request, and that this will allow us to keep backward
compatibility
with 10+ years of code/user habits, but the result will most likely be
confusing.

It will be even harder if we have to remove the usage of `?`/`??`[4].

I also want to note that the use of `?`/`??` is not present to just being or end
of identifiers as it can also be used use to search for names:

> In [1]: *int*?
> FloatingPointError
> int
> print

But this usage is not as widespread as extracting help about objects,
and seem less relevant, though I'm not sure:

> In [10]: ?Float*Error
> FloatingPointError
>
> In [12]: Uni*Error?
> UnicodeDecodeError
> UnicodeEncodeError
> UnicodeError
> UnicodeTranslateError

Please take these fact into consideration when making a
decision/writing the Pep.

Thanks,
--
M

[1]: That's one of the only book for which I have (legally) the
sources, and that I bother to grepped through.
[2]: http://www.pythonforbeginners.com/basics/ipython-a-short-introduction
[3]: http://imgur.com/d0Vs7Xr
[4]: I'll have to hire a body guard to prevent people to pursue me to
the end of the earth with a chainsaw. I'm sure you know that feeling.