Missing exceptions in PEP 3107
Christoph Zwerschke
exceptions are not mentioned there. I think it would be helpful if one
could specify which exceptions can be raised by a function, similarly to
how it is possible in C++ using the "throw" clause. The syntax would be
something like this:
def foo(a: expr, b: expr = 5) raises expr -> expr:
The expr in that "raises" clause should be a list of Exceptions.
Having the list of possible exceptions as annotation alone would be
already helpful. Of course it could be also discussed whether Python
should check that the function really raises only these exceptions (as
in C++), or we could even have checked exceptions (as in Java, but this
seems to be a controversial issue).
Has this already been discussed, or is it in a different PEP?
-- Christoph
Matimus
Keep in mind that annotations are just a way of associating some
information with the parameters or a function. There is a special
parameter called `return` to help associate information with the
return value. Whether that information is used to describe the types
of the function parameters, how they are used, or something completely
different is up to the application that uses them.
When you say:
> The expr in that "raises" clause should be a list of Exceptions.
You are clearly confusing the annotation feature with a possible
application of the annotation feature. Annotation could be used for
many different applications besides type safety.
Annotation simply creates a dictionary. The name `return` was chosen
for the return value because it _is_ a keyword and therefore could not
conflict with the name of any of the parameters. Using "raises" would
mean that we would have to introduce the name "raises" as a new
keyword. It would be better just to use they existing keyword "raise".
With all of that being said, a package or application that uses
annotation could simply use the data-structure associated with
"return" to also contain exception information. That might not seem
intuitive, but keep in mind that the value associated with "return" in
the associations dictionary is going to be a special case anyway.
def foo(a: "a info", b: "b info") -> "return info", "exception info":
return "hello world"
Matt
Christoph Zwerschke
>> The expr in that "raises" clause should be a list of Exceptions.
>
> You are clearly confusing the annotation feature with a possible
> application of the annotation feature. Annotation could be used for
> many different applications besides type safety.
Sorry, I wanted to say "*could* be a list of Exceptions". Though this is
the most obvious application. In the end, the annotations need to be
given a semantical meaning anyway.
> Annotation simply creates a dictionary. The name `return` was chosen
> for the return value because it _is_ a keyword and therefore could not
> conflict with the name of any of the parameters. Using "raises" would
> mean that we would have to introduce the name "raises" as a new
> keyword. It would be better just to use they existing keyword "raise".
Yes, it later also occured to me that it ought to be an existing
keyword, i.e. "raise" (or maybe "except"). That's porbably the reason
why it is "throw" in C++, and not "throws".
> With all of that being said, a package or application that uses
> annotation could simply use the data-structure associated with
> "return" to also contain exception information. That might not seem
> intuitive, but keep in mind that the value associated with "return" in
> the associations dictionary is going to be a special case anyway.
>
> def foo(a: "a info", b: "b info") -> "return info", "exception info":
> return "hello world"
That would be possible. But I still think it makes sense to separate
them, like so:
def foo(a: "a info", b: "b info") -> "ret info" raise "exc info":
return "hello world"
And then the annotation dictionary would contain another key "raise"
containing the exc info. This cannot conflict with the name of any other
parameter either.
Maybe the following syntax would be even more intuitive:
def foo(a: "a info", b: "b info") return "ret info" raise "exc info":
return "hello world"
I don't know how determined the "->" syntax is already.
-- Christoph
Duncan Booth
> That would be possible. But I still think it makes sense to separate
> them, like so:
>
> def foo(a: "a info", b: "b info") -> "ret info" raise "exc info":
> return "hello world"
>
> And then the annotation dictionary would contain another key "raise"
> containing the exc info. This cannot conflict with the name of any other
> parameter either.
>
If you really want this then you can use a decorator to insert a 'raise'
key into the annotations:
@raises("exc info")
def foo(a: "a info", b: "b info") -> "ret info":
return "hello world"
> I don't know how determined the "->" syntax is already.
Consider the syntax set in concrete. The meaning of the annotations on the
other hand is completely up for grabs.
Christoph Zwerschke
> If you really want this then you can use a decorator to insert a 'raise'
> key into the annotations:
Well, yes, but wasn't the whole point of PEP 3107 to get rid of such
decorators and provide a single standard way of specifying this kind of
info instead?
>> I don't know how determined the "->" syntax is already.
>
> Consider the syntax set in concrete. The meaning of the annotations on the
> other hand is completely up for grabs.
Yes, as far as I understand this is an experiment how people are using
the annotations, and then later this may get standardized as well.
But maybe the PEP should then at least mention what's the currently
recommended way to make annotations about thrown exceptions.
-- Christoph
Duncan Booth
> But maybe the PEP should then at least mention what's the currently
> recommended way to make annotations about thrown exceptions.
>
There is no currently recommended way to make such annotations, so how
could the PEP mention it?
I think the problem you are going to face is that almost any function could
potentially throw almost any exception. That may mean it is a good idea to
document what exceptions you expect to be thrown, but it also means you
cannot expect those annotations to be anything other than documentation.
Christoph Zwerschke
> There is no currently recommended way to make such annotations, so how
> could the PEP mention it?
Then it could mention the fact that there is currently no recommended
way (and maybe make some suggestions, like those given by you).
Carl Banks
I think you're missing the point here. PEP 3017 is policy-neutral: it
describes a mechanism to annotate functions and arguments, and that's
it.
IOW, there is currently no recommended way to do *anything* with
annotations(**). That is entirely left up to users and third-party
packages, and the PEP goes out of its way to disclaim all authority on
policy. The following quote from the PEP sums it up well:
"Following from point 2, this PEP makes no attempt to introduce any
kind of standard semantics, even for the built-in types. This work
will be left to third-party libraries."
Your concern is misplaced; it just doesn't belong in the PEP.
"So", you might ask, "where does it belong then?"
The answer is probably "nowhere". Since annotations are intended to
be used by third party packages, those packages will define the
semantics of the annotations, and the recommendations would only be
applicable to users of that package, and not to Python users in
general.
It might come to pass that someday a different PEP will be written to
standarize stuff like this, but that usually only happens after the
community has had time to explore the problem domain for awhile (cf.
WSGI).
Carl Banks
(**) - Actually there is a minor policy recommendation: that the pydoc
and inspect module learn to understand and display the annotations.
Matimus
>
> def foo(a: "a info", b: "b info") return "ret info" raise "exc info":
> return "hello world"
>
> I don't know how determined the "->" syntax is already.
That seems much more intuitive and extensible. The "->" syntax has
always bothered me. The main issue I see with it though is that it
might be confusing. Consider:
def foo(a, b) return 0:
return a + b
A person reading the code might be tempted to read the annotation and
think that it is the body. Maybe not a huge problem, but definitely
something that will come up occasionally.
> Consider the syntax set in concrete.
Why? Python syntax is always changing. If we can think of a better way
to do something, then there is no better time than today to bring it
up.
Having said that, I like the decorator idea too:
> @raises("exc info")
> def foo(a: "a info", b: "b info") -> "ret info":
> return "hello world"
And to this:
> Well, yes, but wasn't the whole point of PEP 3107 to get rid of such
> decorators and provide a single standard way of specifying this kind of
> info instead?
Maybe, but I think it also does two more things: 1. creates a standard
location for storing annotations, and 2. Keeps you from violating DRY
(http://en.wikipedia.org/wiki/DRY).
For instance:
@parameters(a="a info", b="b info")
@raises("exception info")
@returns("return info")
def foo(a, b):
pass
a and b are mentioned in both the definition and the "parameters"
decorator. This violates DRY since a change to the definition will
also require a change to the parameters decorator call. One could
argue that you could make the parameters decorator inspect the
function and apply the annotations positionally. That doesn't really
eliminate the problem, just muddles it. Moving or changing parameters
is still going to result in the need to change code in multiple
locations. The positional case is almost worse in that it will usually
result in the same amount of work, while being less explicit.
Using a single decorator for exception info (or even return info) does
not violate either of the two stated benefits. The exception
information would go into the standard annotations dictionary. The
raises decorator does not violate DRY any more or less than it would
if added to the language syntax.
Matt
Christoph Zwerschke
> Christoph wrote:
>> Maybe the following syntax would be even more intuitive:
>>
>> def foo(a: "a info", b: "b info") return "ret info" raise "exc info":
>> return "hello world"
>
> always bothered me. The main issue I see with it though is that it
> might be confusing. Consider:
>
> def foo(a, b) return 0:
>
> return a + b
>
> A person reading the code might be tempted to read the annotation and
> think that it is the body. Maybe not a huge problem, but definitely
> something that will come up occasionally.
Yes, that's a drawback; and the same problem for a "raise" clause.
>> Well, yes, but wasn't the whole point of PEP 3107 to get rid of such
>> decorators and provide a single standard way of specifying this kind of
>> info instead?
>
> Maybe, but I think it also does two more things: 1. creates a standard
> location for storing annotations, and 2. Keeps you from violating DRY
> (http://en.wikipedia.org/wiki/DRY).
> Using a single decorator for exception info (or even return info) does
> not violate either of the two stated benefits. The exception
> information would go into the standard annotations dictionary. The
> raises decorator does not violate DRY any more or less than it would
> if added to the language syntax.
That's a valid point, but as you already mentioned, the same applies to
the return value. In my opinion it is desirable that either both return
value and exceptions get a special syntax, or both must be described
using decorators.
-- Christoph
Christoph Zwerschke
> I think you're missing the point here. PEP 3017 is policy-neutral:
> it describes a mechanism to annotate functions and arguments,
> and that's it.
That's not quite true: PEP 3017 describes a mechanism for annotating
function parameters *and return values*, and my point was why it does
not provide a mechanism for annotating thrown exceptions, since I see
info on thrown exceptions in the same category as info on return values.
> IOW, there is currently no recommended way to do *anything* with
> annotations(**). That is entirely left up to users and third-party
> packages, and the PEP goes out of its way to disclaim all authority on
> policy. The following quote from the PEP sums it up well:
>
> "Following from point 2, this PEP makes no attempt to introduce any
> kind of standard semantics, even for the built-in types. This work
> will be left to third-party libraries."
That's right, the PEP does not say how the information is to be
*interpreted*, but it does say what the information is *about* (e.g. a
certain parameter or the return value). In the same way, you could make
annotations about thrown exceptions without actually defining how this
information should be interpreted.
> Your concern is misplaced; it just doesn't belong in the PEP.
Call me slow on the uptake, but I still think that if you discuss
function annotations and ways of specifying info on return values, then
ways of specifying thrown exceptions belong into the same discussion. I
understand if you say that for organizational/workflow reasons, the PEP
cannot be altered or supplemented any more since it has reached the
"final" status. But as regards contents, I still think it belongs there.
> (**) - Actually there is a minor policy recommendation: that the pydoc
> and inspect module learn to understand and display the annotations.
What do you mean with "understand" here when no semantics exists? Do you
mean that they should just be able to parse and display them, or that
they should start to define the semantics?
-- Christoph
Terry Reedy
Christoph Zwerschke wrote:
> That's not quite true: PEP 3017 describes a mechanism for annotating
> function parameters *and return values*, and my point was why it does
> not provide a mechanism for annotating thrown exceptions, since I see
> info on thrown exceptions in the same category as info on return values.
I would agree... but...
The problem is that code that uses a function hardly cares whether an
exception that replaces the normal return is raised explicitly, by a
syntax operation (and these are not yet completely documented, though
perhaps they should be), or by a function called within the function.
tjr
Carl Banks
> Carl Banks wrote:
> > IOW, there is currently no recommended way to do *anything* with
> > annotations(**). That is entirely left up to users and third-party
> > packages, and the PEP goes out of its way to disclaim all authority on
> > policy. The following quote from the PEP sums it up well:
> >
> > "Following from point 2, this PEP makes no attempt to introduce any
> > kind of standard semantics, even for the built-in types. This work
> > will be left to third-party libraries."
>
> That's right, the PEP does not say how the information is to be
> *interpreted*, but it does say what the information is *about* (e.g. a
> certain parameter or the return value).
No it doesn't. The PEP says what the information's typical use case
is, not what purpose it's mandated for.
You are free to use it for other things. For example, the following
usage is obvious and sensible (insofar as listing exceptions is
sensible):
def f(x : int) -> int, raises(TypeError)
Think of the return value annotation as more of a function outcome
annotation. The following are all reasonable uses of it:
def f() -> no_return
def g() -> modifies_argument
def h() -> modifies_global_config
def j() -> raises(StopIteration)
The designers of Python could have added syntax speficially to list
exceptions, but the fact is, the return value annotation can handle
this perfectly well without needing specific syntax. Python does not
like to add unnecessary syntax.
Carl Banks
Christoph Zwerschke
> You are free to use it for other things. For example, the following
> usage is obvious and sensible (insofar as listing exceptions is
> sensible):
>
> def f(x : int) -> int, raises(TypeError)
>
> Think of the return value annotation as more of a function outcome
> annotation.
That's fine, but then this should be mentioned in the PEP3107. Instead
it says explicitly that the "->" syntax is for "the type of a function's
return value". If it's intended to be used in a more general way like
you suggested, then the wording used in the PEP is misleading. (The
wording "*the* type is problematic anyway, since a Python function can
return different types in different situations.)
It's still unclear for me whether annotations on thrown exceptions are
included in PEP3107 (as per your suggestion), or are not covered by
PEP3107 at all (as Duncan suggests), or whether this question has just
not yet been settled.
-- Christoph
Christoph Zwerschke
> I would agree... but...
> The problem is that code that uses a function hardly cares whether an
> exception that replaces the normal return is raised explicitly, by a
> syntax operation (and these are not yet completely documented, though
> perhaps they should be), or by a function called within the function.
I often read that argument that info on thrown exceptions does not
matter in Python, but I beg to differ. Just as a simple and well-known
example, it is absolutely important to know that the str.index() method
throws a ValueError if nothing is found, while the str.find() method
should never throw a ValueError.
-- Christoph
Carl Banks
> Carl Banks schrieb:
>
> > You are free to use it for other things. For example, the following
> > usage is obvious and sensible (insofar as listing exceptions is
> > sensible):
>
> > def f(x : int) -> int, raises(TypeError)
>
> > Think of the return value annotation as more of a function outcome
> > annotation.
>
> That's fine, but then this should be mentioned in the PEP3107.
Evidently the thing that really needs to be mentioned in the PEP is
"don't be pedantic".
> Instead
> it says explicitly that the "->" syntax is for "the type of a function's
> return value". If it's intended to be used in a more general way like
> you suggested, then the wording used in the PEP is misleading. (The
> wording "*the* type is problematic anyway, since a Python function can
> return different types in different situations.)
For some reason, you keep trying to interpret the PEP's description as
policy.
Once again: the PEP disclaims all authority on policy. If the PEP
intended you to use the annotations only for what it said the
annotations were for, then it wouldn't have disclaimed that authority.
> It's still unclear for me whether annotations on thrown exceptions are
> included in PEP3107 (as per your suggestion), or are not covered by
> PEP3107 at all (as Duncan suggests), or whether this question has just
> not yet been settled.
According to the PEP you can do anything you want with the
annotations. PEP 3107 does not specify policy. There is no question
to settle.
If it bothers you that much, go file a bug report. Someone might even
change it. But it's nothing but needless pedantry.
Carl Banks
Christoph Zwerschke
> If it bothers you that much, go file a bug report. Someone might even
> change it. But it's nothing but needless pedantry.
Has my "de" domain inspired you to rant about "pedantry"? No, it does
not bother me that much. I just thought the PEP could be clearer here
and explicitly mention the thrown exceptions as a use case instead of
only speaking about return values. Of course, if you're Dutch, it may be
obvious, but for a dimwitted German things are not always obvious.
-- Christoph
Carl Banks
> Carl Banks wrote:
> > If it bothers you that much, go file a bug report. Someone might even
> > change it. But it's nothing but needless pedantry.
>
> Has my "de" domain inspired you to rant about "pedantry"?
Actually, no.
Carl Banks
P.S. My great-grandfather's surname was "von Bank" when he emigrated
to America.