[Python-ideas] Inadequate error reporting during function call setup stage

14 views
Skip to first unread message

Paul Sokolovsky

unread,
Feb 21, 2021, 12:15:24 PM2/21/21
to Python-Dev, python...@python.org
Hello,

Here's example:

Traceback (most recent call last):
File "pseudoc_tool.py", line 91, in <module>
first_class_function_value(func, **pass_params)
TypeError: print() got an unexpected keyword argument 'noann'


Ok, which "print" do you mean, dear CPython? I have a dozen of print
functions (mostly methods) around. So I now need to reduce to printf
debugging to find which one is actually meant. And what if that happens
in 3rd-party library installed in /usr/lib? On a production system?

The most interesting fact is that despite functions has always been
first-class values and 30-years language history, such issues are still
there.

I'm fixing this right away in my Pycopy dialect, because while it's
minimalist and I tremble over each half-byte added, but it's Python
first of all, and Python means good error reporting.

Well, actually I don't fix it right away, but work around, put putting
source loc in TypeError's error message. And here's an idea of what the
real fix should be:

The fact that CPython (and Pycopy) sets up function's frame "outside"
the function execution is an implementation detail. The model of
execution should be: there's a CALL_FUNCTION/CALL_FUNCTION_KW opcode,
and after it executes, the context switches to the target function, and
that's *when* argument checking happens. That would give the desired
behavior that the last entry in traceback is the actual function
which was called and which failed parameter checks.

Again, it's an implementation detail if some project doesn't do it like
that, and performs param checking before actually pointing VM at the
new bytecode function (from which location traceback will be
automatically constructed). If it does that, it just still need to
append an additional traceback entry pointing to the beginning of the
target function.

I now implemented that too, and now everything makes sense:

Traceback (most recent call last):
File "pseudoc_tool.py", line 91, in <module>
File "../xforms.py", line 25, in print
TypeError: unexpected keyword argument 'noann'


--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/VY3QAU2TEXOPELGDS7RTQOX4357QWILR/
Code of Conduct: http://python.org/psf/codeofconduct/

Peter Otten

unread,
Feb 22, 2021, 4:47:41 AM2/22/21
to Terry Reedy, pytho...@python.org, python...@python.org
On 21/02/2021 23:06, Terry Reedy wrote:
> On 2/21/2021 12:04 PM, Paul Sokolovsky wrote:
>
>> Traceback (most recent call last):
>>    File "pseudoc_tool.py", line 91, in <module>
>>      first_class_function_value(func, **pass_params)
>> TypeError: print() got an unexpected keyword argument 'noann'
>
> This is not typical behavior in current Python (3.8+).

The way I understand it's not about print(), it's about
disambiguating multiple functions with the same name.
Example:

PS > type .\ambiguous_names.py
import random

def do_stuff():
pass

f = do_stuff

def do_stuff(a, b):
pass

g = do_stuff

random.choice([f, g])(42)


PS > py .\ambiguous_names.py
Traceback (most recent call last):
File "...\ambiguous_names.py", line 13, in <module>
random.choice([f, g])(42)
TypeError: do_stuff() missing 1 required positional argument: 'b'

The traceback gives no clue which of the two do_stuff() functions caused
the error, you have to check both implementations.

If that is a comman problem one might consider including module name and
co_firstlineno in the message, or at least adding the relevant
do_stuff() function to the exception's args.

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/E7OUHLWXP3NHEM3UZOBWP5AQRSFL5RDO/

Paul Sokolovsky

unread,
Feb 22, 2021, 5:26:59 AM2/22/21
to Peter Otten, Terry Reedy, pytho...@python.org, python...@python.org
Hello,

On Mon, 22 Feb 2021 10:44:19 +0100
Peter Otten <__pet...@web.de> wrote:

> On 21/02/2021 23:06, Terry Reedy wrote:
> > On 2/21/2021 12:04 PM, Paul Sokolovsky wrote:
> >
> >> Traceback (most recent call last):
> >>    File "pseudoc_tool.py", line 91, in <module>
> >>      first_class_function_value(func, **pass_params)
> >> TypeError: print() got an unexpected keyword argument 'noann'
> >
> > This is not typical behavior in current Python (3.8+).
>
> The way I understand it's not about print(), it's about
> disambiguating multiple functions with the same name.
> Example:
>
> PS > type .\ambiguous_names.py
> import random
>
> def do_stuff():
> pass
>
> f = do_stuff
>
> def do_stuff(a, b):
> pass
>
> g = do_stuff
>
> random.choice([f, g])(42)

Thanks, that's exactly what I meant, and a repro with random "roulette"
is also what I had in mind, I just didn't get to it yet ;-).

> PS > py .\ambiguous_names.py
> Traceback (most recent call last):
> File "...\ambiguous_names.py", line 13, in <module>
> random.choice([f, g])(42)
> TypeError: do_stuff() missing 1 required positional argument: 'b'
>
> The traceback gives no clue which of the two do_stuff() functions
> caused the error, you have to check both implementations.
>
> If that is a comman problem one might consider including module name
> and co_firstlineno in the message, or at least adding the relevant
> do_stuff() function to the exception's args.

As my original message argues, that's a workaround. Python tracebacks
already have places where they show source file and line number -
namely, the individual traceback entries. So, instead of cramming that
info into the exception message, there should be additional last (latest
in the order of execution) traceback entry, pointing to the exact
function which had parameter mismatch. As I mentioned, I implemented
that in my Python dialect, which happened to have exactly the same
problem (code is not based on CPython).

It looks like:

Traceback (most recent call last):
File "pseudoc_tool.py", line 91, in <module>
File ".../xforms.py", line 25, in print
TypeError: unexpected keyword argument 'noann'

- that makes clear that it's "print" function of "xforms.py" module,
line 25, which got an unexpected keyword argument.


--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/7SJTTAB7TZQYZ6WSRAAPSLLNMRF2OP7F/

Barry Scott

unread,
Feb 22, 2021, 2:50:49 PM2/22/21
to Paul Sokolovsky, Peter Otten, Terry Reedy, Python-Dev, python...@python.org


On 22 Feb 2021, at 10:15, Paul Sokolovsky <pmi...@gmail.com> wrote:

It looks like:

Traceback (most recent call last):
 File "pseudoc_tool.py", line 91, in <module>
 File ".../xforms.py", line 25, in print
TypeError: unexpected keyword argument 'noann'

- that makes clear that it's "print" function of "xforms.py" module,
line 25, which got an unexpected keyword argument.

You are proposing to fake a stack frame that I have to know is not a stack frame but is in fact the location of the function in the exception?
I'm -1 on that as its confusing.

Having checked that its python code and not a C extension function you could use the info
in fn.__code__ to get the filename and line of where the function is defined and put that info into the exception.

Example of the info:
| >>> os.path.join.__code__
<code object join at 0x7fb19aebc7c0, file "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/posixpath.py", line 71>

I use repr(fn.__code__) a lot when debugging complex code.

Barry

Paul Sokolovsky

unread,
Feb 22, 2021, 3:20:14 PM2/22/21
to Barry Scott, Peter Otten, Terry Reedy, Python-Dev, python...@python.org
Hello,

On Mon, 22 Feb 2021 19:47:04 +0000
Barry Scott <ba...@barrys-emacs.org> wrote:

> > On 22 Feb 2021, at 10:15, Paul Sokolovsky <pmi...@gmail.com> wrote:
> >
> > It looks like:
> >
> > Traceback (most recent call last):
> > File "pseudoc_tool.py", line 91, in <module>
> > File ".../xforms.py", line 25, in print
> > TypeError: unexpected keyword argument 'noann'
> >
> > - that makes clear that it's "print" function of "xforms.py" module,
> > line 25, which got an unexpected keyword argument.
>
> You are proposing to fake a stack frame that I have to know is not a
> stack frame but is in fact the location of the function in the
> exception?

No, I'm proposing to stop faking lack of the last stack frame due to
CPython's implementation details. See the original message for more
info.

> I'm -1 on that as its confusing.
>
> Having checked that its python code and not a C extension function
> you could use the info in fn.__code__ to get the filename and line of
> where the function is defined and put that info into the exception.

Could use crystal ball, even.

>
> Example of the info:
> | >>> os.path.join.__code__
> <code object join at 0x7fb19aebc7c0, file
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/posixpath.py",
> line 71>
>
> I use repr(fn.__code__) a lot when debugging complex code.
>
> Barry
>



--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/FB65TFWGZEBENGT3W2X7RBPY63RINU25/

Ned Batchelder

unread,
Feb 22, 2021, 5:39:21 PM2/22/21
to python-ideas

On 2/22/21 3:06 PM, Paul Sokolovsky wrote:
>
> No, I'm proposing to stop faking lack of the last stack frame due to
> CPython's implementation details. See the original message for more
> info.

I'm trying to understand what last stack frame (or lack of a last stack
frame, or the faking of a lack of a last stack frame) you are talking
about.  Your original message shows some tracebacks, but doesn't have
the code that produced them.  It's hard to understand what you are
referring to.

--Ned.
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/KTBIYQDPHOHNYGTWWXT2KRODRHZDZWQX/

Stestagg

unread,
Feb 22, 2021, 6:10:37 PM2/22/21
to Ned Batchelder, python-ideas
My take on this is that it's actually a valid suggestion, albeit wrapped in some unhelpful language.

This is a more common example, in my opinion:

=== fileA.py ===

class MyClass:
    def do_thing(self, arg_a, arg_b):
        pass

=== fileB.py ===
...
class MySubClass(MyClass):
    pass

=== main.py ===
...
MySubclass().do_thing(1)

If you run the above (with trivial imports filled in etc.)
Then the traceback gives:

Traceback (most recent call last):
  File "c.py", line 3, in <module>
    B().do_thing()
TypeError: do_thing() takes exactly 3 arguments (1 given)

So Python has identified that a function 'do_thing' is being called incorrectly, but /where/ is do_thing defined?   This problem gets much harder if there are multiple definitions of 'do_thing' in the codebase, as alluded to in the original mail.

Paul's suggestion is that python should add the source location of the function being called into the error message somewhere.

The "it is/isn't a stack frame" issue is one possible solution, and comes down to a potential *semantic only* disagreement on how functions are called with python.

If you take traditional runtime-less languages, then the responsibility for unpacking and handling arguments is considered part of the function being called.
With C, for example, typically the compiler inserts a header (can't remember the better term for this right now)  at the start of a function, which moves the function arguments around as needed, so any failure match the arguments originates from within the called function (this is even more obvious with VA_ARGS for example).

Obviously, in languages with a runtime, such as python, the responsibility for matching arguments with function signatures can sit outside of the runtime function call context, and runtime-internal machinery information is not typically included in stack-traces.  Therefore, it appears (although I don't think this was part of any specific plan?) that the error originates from the call site (where the function is being called from) because this is the last point that was executing any python code.

So, the suggestion, as I understand it, is that a small conceptual shift is made in how we think about the mechanics of calling a function within python, such that the assignment and binding of function arguments are conceptually performed 'inside' the function being called (I assume you could equate this to the function 'def' line being executed).  If we do this, then it becomes easy to consider the TypeError as originating from within the called function, and therefore python will print the exact location of the function that's being called *as well as* the location of the call site (in the frame above).  Internally, the python runtime would have to go out of its way to  enter this 'fake' frame before generating the exception as, in reality the frame isn't entered (as I understand it) until after the arguments have been assigned correctly to the signature.  But this seems like it should be relatively simple and non-invasive to do.

The above makes sense to me, apologies if it's not very understandable to others.

Thanks

Steve


Stestagg

unread,
Feb 22, 2021, 6:12:14 PM2/22/21
to python-ideas
To avoid confusion, the traceback below has been updated here with correct names:

On Mon, Feb 22, 2021 at 11:07 PM Stestagg <stes...@gmail.com> wrote:
My take on this is that it's actually a valid suggestion, albeit wrapped in some unhelpful language.

This is a more common example, in my opinion:

=== fileA.py ===

class MyClass:
    def do_thing(self, arg_a, arg_b):
        pass

=== fileB.py ===
...
class MySubClass(MyClass):
    pass

=== main.py ===
...
MySubclass().do_thing(1)

If you run the above (with trivial imports filled in etc.)
Then the traceback gives:

Traceback (most recent call last):
  File "main.py", line 3, in <module>
    MySubClass().do_thing(1)
TypeError: do_thing() takes exactly 3 arguments (2 given)

Chris Angelico

unread,
Feb 22, 2021, 8:32:55 PM2/22/21
to python-ideas
On Tue, Feb 23, 2021 at 10:10 AM Stestagg <stes...@gmail.com> wrote:
> So Python has identified that a function 'do_thing' is being called incorrectly, but /where/ is do_thing defined? This problem gets much harder if there are multiple definitions of 'do_thing' in the codebase, as alluded to in the original mail.
>
> Paul's suggestion is that python should add the source location of the function being called into the error message somewhere.
>

Thank you for explaining. I'm glad I wasn't the only one confused by
the original post :)

I think this is a nice-to-have, rather than being a serious bug to be
fixed. When you get an error about a function call, it's entirely
possible that the target function is the problem, or that the wrong
function is being referenced; but you can get similar problems with
all kinds of mismatches (like "a, b, c = thing" and getting a
ValueError - maybe you'd need to check the line where 'thing'
originated).

If this can be done easily, great, but otherwise it might be the
domain of traceback enhancement tools rather than the core language.

Does execution ever (in the normal case) actually hit the 'def' line?
If so, I wouldn't be averse to having a traceback line mentioning it.
But if (as I suspect) tracing successful execution wouldn't hit that
line, then it doesn't seem right to feign execution of it. My
understanding of the def statement is that it runs ONLY when the
function is defined; when the function's called, you go straight into
the body.

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/6G6Q45XYZOFA7BMHDLJUO4R6AX5UJVDM/

Paul Sokolovsky

unread,
Feb 23, 2021, 12:59:45 AM2/23/21
to Ned Batchelder, python-ideas
Hello,

On Mon, 22 Feb 2021 17:32:23 -0500
Ned Batchelder <n...@nedbatchelder.com> wrote:

> On 2/22/21 3:06 PM, Paul Sokolovsky wrote:
> >
> > No, I'm proposing to stop faking lack of the last stack frame due to
> > CPython's implementation details. See the original message for more
> > info.
>
> I'm trying to understand what last stack frame (or lack of a last
> stack frame, or the faking of a lack of a last stack frame) you are
> talking about.  Your original message shows some tracebacks, but
> doesn't have the code that produced them.  It's hard to understand
> what you are referring to.

Well, if you looked at that stack trace, you saw the code:

Traceback (most recent call last):
File "pseudoc_tool.py", line 91, in <module>
first_class_function_value(func, **pass_params)
TypeError: print() got an unexpected keyword argument 'noann'

So yes, the code is:

---
first_class_function_value(func, **pass_params)
---

That's the "calling code". And the rest of the code? But's that's the
whole point, that the current CPython's error reporting doesn't tell me
where it is! It just tells me the function name, and go make a
full-text search thru the entire sys.path to find its location,
especially if there're many functions of that name.



--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/6GFRPZSWTRZPJQKMWKBJVVPHD7X2ZMLO/

Paul Sokolovsky

unread,
Feb 23, 2021, 1:36:42 AM2/23/21
to Chris Angelico, python-ideas
Hello,

On Tue, 23 Feb 2021 12:31:37 +1100
Chris Angelico <ros...@gmail.com> wrote:

> On Tue, Feb 23, 2021 at 10:10 AM Stestagg <stes...@gmail.com> wrote:
> > So Python has identified that a function 'do_thing' is being called
> > incorrectly, but /where/ is do_thing defined? This problem gets
> > much harder if there are multiple definitions of 'do_thing' in the
> > codebase, as alluded to in the original mail.
> >
> > Paul's suggestion is that python should add the source location of
> > the function being called into the error message somewhere.
>
> Thank you for explaining. I'm glad I wasn't the only one confused by
> the original post :)
>
> I think this is a nice-to-have, rather than being a serious bug to be
> fixed.

It's not a serious bug, it's "astonishing overlook". Astonishing, as it
was there for so many years, unpatched. And of course, it's rare
enough. But when it happens, you're in the maze.

I spend much time communicating with Python critics (well, haters), and
sometimes read quite funny criticisms like: error reporting
(stacktraces) in Python is poor. I always considered that "random last
resort nitpick", as error reporting in Python has always been much
better than in other scripting languages (I guess by now everyone
caught up).

But now imagine error like that happens in some 3rd-party library.
Which happened to do some first-class function/higher order
programming. With functions from another 3rd-party library. Which
released a new version with "slightly updated" API. All that happening
in production. Someone who ever caught such (or similar) case would
forever get imprint of "Python has poor error reporting". Because just
imagine what you report to the maintainer of the first 3rd-party lib:
"Your lib calls something wrong in file X line Y" - "Ok, let's look
into that. What does it call?" - "Umm, I don't know, something called
'foo'" (or "<lambda>").

> When you get an error about a function call, it's entirely
> possible that the target function is the problem, or that the wrong
> function is being referenced; but you can get similar problems with
> all kinds of mismatches (like "a, b, c = thing" and getting a
> ValueError - maybe you'd need to check the line where 'thing'
> originated).

Good point. But right analogy here is: suppose error happens in "a, b,
c = thing". But Python doesn't show that line in the backtrace, it stops
1 entry before it. Go figure now.

Because that's the right analogy of what happens with a function - in
the function "prolog", there's a series of assignments:
"formal_paramX = actual_argX". Conceptually, those *are* in the
function context (or you would not be able to reference function's
formal params).

> If this can be done easily, great, but otherwise it might be the
> domain of traceback enhancement tools rather than the core language.

For Pycopy it was easy. I will look into CPython one of these weeks.

>
> Does execution ever (in the normal case) actually hit the 'def' line?
> If so, I wouldn't be averse to having a traceback line mentioning it.
> But if (as I suspect) tracing successful execution wouldn't hit that
> line, then it doesn't seem right to feign execution of it. My
> understanding of the def statement is that it runs ONLY when the
> function is defined; when the function's called, you go straight into
> the body.

For another analogy, you can look at how native functions are defined
and where they check params, and from where exceptions originate there
(by obvious reasons, Python backtraces don't show C source code
function/linenos ;-)).

>
> ChrisA

[]
--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/FB46HVUMI4KYBQW3URH75WV7VJBI5U3G/

Chris Angelico

unread,
Feb 23, 2021, 1:55:44 AM2/23/21
to python-ideas
On Tue, Feb 23, 2021 at 5:33 PM Paul Sokolovsky <pmi...@gmail.com> wrote:
> Because that's the right analogy of what happens with a function - in
> the function "prolog", there's a series of assignments:
> "formal_paramX = actual_argX". Conceptually, those *are* in the
> function context (or you would not be able to reference function's
> formal params).

But they aren't actual lines of code. Your analogy breaks down because
you're trying to put a line into a backtrace for a mythical and
conceptual action that happens as part of the function call.

That's why I dispute that this is an "astonishing oversight". It would
be nice to have the extra information, but given that Python and
Python programmers have survived for thirty years without it, I don't
think it's nearly as serious as you're implying.

> For another analogy, you can look at how native functions are defined
> and where they check params, and from where exceptions originate there
> (by obvious reasons, Python backtraces don't show C source code
> function/linenos ;-)).

Yes, or you could look at CPython byte code and how it does a bunch of
stack operations, but some of them don't actually happen. Looking at
the implementation, especially in a completely different language,
doesn't really help here :)

The way I see it, this is perfect as an "additional information"
field. For instance, when I compile certain buggy C programs, gcc
tells me things like this:

demo.c: In function ‘main’:
demo.c:4:9: warning: passing argument 1 of ‘printf’ makes pointer from
integer without a cast [-Wint-conversion]
printf(12345);
^~~~~
In file included from demo.c:1:
/usr/include/stdio.h:332:43: note: expected ‘const char * restrict’
but argument is of type ‘int’
extern int printf (const char *__restrict __format, ...);
~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~

It starts by reporting the error, and then says "note: the thing
you're calling came from over here". It's separate from the error
itself, but provides extremely useful information.

Python doesn't currently have a standardized way to report this, but
IMO that would be a far better way than synthesizing a fake backtrace
entry.

Taking your original example, it might look something like this:

Traceback (most recent call last):
File "pseudoc_tool.py", line 91, in <module>
first_class_function_value(func, **pass_params)
TypeError: print() got an unexpected keyword argument 'noann'

Note: The function is defined here:
File "whatever.py", line 123, in <module>
def print(thingy, thongy, whop):

ChrisA

_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/5Z4JLH76N6OL4DGF4CBD3IY7GQQOL2VR/

Paul Sokolovsky

unread,
Feb 23, 2021, 2:13:36 AM2/23/21
to Chris Angelico, python-ideas
Hello,

On Tue, 23 Feb 2021 17:52:20 +1100
Chris Angelico <ros...@gmail.com> wrote:

> On Tue, Feb 23, 2021 at 5:33 PM Paul Sokolovsky <pmi...@gmail.com>
> wrote:
> > Because that's the right analogy of what happens with a function -
> > in the function "prolog", there's a series of assignments:
> > "formal_paramX = actual_argX". Conceptually, those *are* in the
> > function context (or you would not be able to reference function's
> > formal params).
>
> But they aren't actual lines of code. Your analogy breaks down because
> you're trying to put a line into a backtrace for a mythical and
> conceptual action that happens as part of the function call.

But that's the whole point - the externally visible behavior should
correspond to the conceptual model, even if a particular implementation
does something different underlyingly.

So again, it's CPython's implementation detail that, for bytecode
functions only, it implements:

def foo(params): # foo takes care of its params
...

foo(args)

as:

param_check_and_call(foo, __param_meta__[foo], *args)

And due to this implementation detail raises exceptions from
param_check_and_call().

[]

> The way I see it, this is perfect as an "additional information"
> field.

I started there, and proceeded to where no "additional information"
hacks are needed, and everything fits well into the existing framework.
Come along with me.


[]


--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/CBWMQQYKMOL6JYAESTHFWCJ3Y7H53CDK/

Paul Sokolovsky

unread,
Feb 23, 2021, 2:38:14 AM2/23/21
to Chris Angelico, python-ideas
Hello,

On Tue, 23 Feb 2021 17:52:20 +1100
Chris Angelico <ros...@gmail.com> wrote:

[]

> That's why I dispute that this is an "astonishing oversight". It would
> be nice to have the extra information, but given that Python and
> Python programmers have survived for thirty years without it, I don't
> think it's nearly as serious as you're implying.

Ah, missed to reply to that one. Can judge it by myself: that's
definitely not the first time I see that issue, where there was
"something wrong" when getting exceptions when dealing with first-class
function values. But I didn't even have a good understanding of what's
wrong, and cases were either simple and obvious, or I reduced to print
debugging.

But now I'm working on a project where I consider UX to be important.
And I consider a good UX for Python project not when it catch-all's
exceptions and obfuscates the problem with a generic error message, and
likewise not when there're 5 screenfuls of chained exceptions. I
consider it's when there's human-sized backtrace, clearly pointing at
the problem.

And for that case, I figured that the backtrace is senseless and
doesn't give a user of my program enough information to understand the
issue.

So, I stopped with writing my program, sat and scratched my head to
understand what's wrong, then sat again to try a few different
solutions (the original post should mirror that), until finally came
to one which makes sense to the end user, and doesn't require any
crutches.


I won't be surprised if that's how other Python programmers dealt with
it over decades - some didn't even dig into "what's wrong", "something
is wrong" was enough for them to do in-mind-debugging. The other
gritted teeth and recursed to print debugging, again without thinking
too much what's wrong on the meta-level. And the remaining? Well, they
now curse Python for poor error reporting on reddits and hackernewses.


You can also see that on this very thread - at first, the issue was at
all unclear (even though I tried to write a detailed message, packing
different sides into it). Now issue becomes clear, but (besides the
usual "never heard" response), the thinking is along the lines of
"additional info hacks", which was my first motion either.

Let's see where that leads us. (And that's why I posted to the mailing
list first - chances, a bugs.p.o report would be closed with "discuss
first on the mailing list" suggestion).

[]

--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/OIENWJYHNEUNTU67MLKVRNK4SRPOSIEE/

Ned Batchelder

unread,
Feb 23, 2021, 11:56:21 AM2/23/21
to Paul Sokolovsky, python-ideas
Perhaps instead of fiddling with stack frames, we can solve this problem
in a more straightforward way.  The error message has a description of
the function:  "print()".  What if we extend that description: "print()
at path/to/myprog.py:173".  Then the existing stack traces would have
the information you want, without having to debate the meaning of stack
frames, and where execution happens.

--Ned.
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/2Y3U2F3HS7MTDXGKEELHZAIWWEGEJYSD/

Paul Sokolovsky

unread,
Feb 23, 2021, 12:59:12 PM2/23/21
to Ned Batchelder, python-ideas
Hello,

On Tue, 23 Feb 2021 11:52:57 -0500
That's what I implemented initially, saw this:

File "pseudoc_tool.py", line 91, in <module>
first_class_function_value(func, **pass_params)
TypeError: print() at path/to/myprog.py:173 got an unexpected keyword argument 'noann'

and wondered - why am I messing with the error *message*, if above it is
a backtrace entry, which naturally shows the location information ("File
"pseudoc_tool.py", line 91"). Then all things clicked into their
places.

In my implementation, I didn't need to "fiddle with stack frames", I
just needed to pre-populate the exception object with a first
(shown last) backtrace entry for the called function, and let the usual
processing append to it.

> Then the existing
> stack traces would have the information you want, without having to
> debate the meaning of stack frames, and where execution happens.

Without getting to the root cause, there will be only workarounds of
different level of ugliness. For example, any reasonable JIT would
implement proper semantics - param checking *inside* functions (just
like the C code does now).

>
> --Ned.
>



--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/M4BIP3TEYCQNL44JGOFULFCAIZSN4CL4/

Chris Angelico

unread,
Feb 23, 2021, 1:02:57 PM2/23/21
to python-ideas
On Wed, Feb 24, 2021 at 4:59 AM Paul Sokolovsky <pmi...@gmail.com> wrote:
> Without getting to the root cause, there will be only workarounds of
> different level of ugliness. For example, any reasonable JIT would
> implement proper semantics - param checking *inside* functions (just
> like the C code does now).

Why do you assume that this is "proper semantics"? I'm confused. What
is it that makes function parameter matching inherently part of the
body of the function?

For one thing, that notion is semantically incompatible with multiple
dispatch, where matching parameter numbers or types determines which
function is even going to be called.

ChrisA
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/KOXIVI3KX5ST4HA6ERN5ORKLUGO6G3X7/

Ned Batchelder

unread,
Feb 23, 2021, 1:12:53 PM2/23/21
to python-ideas, Paul Sokolovsky
I don't think you are helping your cause by talking down to people. "any
reasonable JIT" and "proper semantics" are judgemental without adding
anything to the discussion.

You wanted to know what function caused the problem. I proposed a simple
solution that gave you precisely the information you wanted. But you've
called it an ugly workaround for some reason.

--Ned.
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/F3DMUYIVBQQ3UJNJ3L2TXJP5HS47KJT5/

Paul Sokolovsky

unread,
Feb 23, 2021, 1:19:29 PM2/23/21
to Chris Angelico, python-ideas
Hello,

On Wed, 24 Feb 2021 05:01:37 +1100
Chris Angelico <ros...@gmail.com> wrote:

> On Wed, Feb 24, 2021 at 4:59 AM Paul Sokolovsky <pmi...@gmail.com>
> wrote:
> > Without getting to the root cause, there will be only workarounds of
> > different level of ugliness. For example, any reasonable JIT would
> > implement proper semantics - param checking *inside* functions (just
> > like the C code does now).
>
> Why do you assume that this is "proper semantics"? I'm confused. What
> is it that makes function parameter matching inherently part of the
> body of the function?

The same what causes C-level function implementations to do parameter
checking themselves. There's simply nowhere else to put that checking,
and of course nobody calls C functions via an intermediary - that's
hilariously inefficient. All those points apply to JITted functions too
(you JIT to make it fast, then putting a slow bottleneck inbetween,
which will also confuse CPU branch predictors - doesn't make sense).

> For one thing, that notion is semantically incompatible with multiple
> dispatch, where matching parameter numbers or types determines which
> function is even going to be called.

Python doesn't support multiple dispatch. Majority of languages don't
support multiple dispatch. For a well known reason - its benefits vs its
inefficiency ratio is very low (inefficiency is high). So, whichever
runtime system implements multiple dispatch, will need to think about
its issues, including aspects of error reporting, I don't see how
that's relevant to Python discussion.

(Multiple dispatch is also effectively a poorman's, implicit pattern
matching which works only between code blocks factored out into
functions.)

[]

--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/WGBXU3IMUDOHPBJMRPHOVOIODJBYPZWA/

Paul Sokolovsky

unread,
Feb 23, 2021, 1:31:09 PM2/23/21
to Ned Batchelder, python-ideas
Hello,

On Tue, 23 Feb 2021 13:09:39 -0500
Well, keeping repeating "we can put location info into the error
message", disregarding an argument that *backtrace entries* are for
location info - that should be classified as "judgemental" too then?

> You wanted to know what function caused the problem. I proposed a
> simple solution that gave you precisely the information you wanted.
> But you've called it an ugly workaround for some reason.

Because I saw them both (by implementing them both), and even in the
order you suggest, "location info in the message" first, felt not
satisfied, spent more time to implement the alternative, and yes, that
looks both more naturally and more clearly. So, please don't be
judgemental to my suggestion, but consider them both, not just first
which came to your mind (which is also exactly the one which came
to my mind first, peace).

>
> --Ned.
>



--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-ideas mailing list -- python...@python.org
To unsubscribe send an email to python-id...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python...@python.org/message/J6NY4UESFAVG323FZSR7Z3EDOVZP2GKT/
Reply all
Reply to author
Forward
0 new messages