[Python-Dev] Inadequate error reporting during function call setup stage

19 views
Skip to first unread message

Paul Sokolovsky

unread,
Feb 21, 2021, 12:08:09 PM2/21/21
to Python-Dev, python...@python.org
Hello,

Here's example:

Traceback (most recent call last):
File "pseudoc_tool.py", line 91, in <module>
first_class_function_value(func, **pass_params)
TypeError: print() got an unexpected keyword argument 'noann'


Ok, which "print" do you mean, dear CPython? I have a dozen of print
functions (mostly methods) around. So I now need to reduce to printf
debugging to find which one is actually meant. And what if that happens
in 3rd-party library installed in /usr/lib? On a production system?

The most interesting fact is that despite functions has always been
first-class values and 30-years language history, such issues are still
there.

I'm fixing this right away in my Pycopy dialect, because while it's
minimalist and I tremble over each half-byte added, but it's Python
first of all, and Python means good error reporting.

Well, actually I don't fix it right away, but work around, put putting
source loc in TypeError's error message. And here's an idea of what the
real fix should be:

The fact that CPython (and Pycopy) sets up function's frame "outside"
the function execution is an implementation detail. The model of
execution should be: there's a CALL_FUNCTION/CALL_FUNCTION_KW opcode,
and after it executes, the context switches to the target function, and
that's *when* argument checking happens. That would give the desired
behavior that the last entry in traceback is the actual function
which was called and which failed parameter checks.

Again, it's an implementation detail if some project doesn't do it like
that, and performs param checking before actually pointing VM at the
new bytecode function (from which location traceback will be
automatically constructed). If it does that, it just still need to
append an additional traceback entry pointing to the beginning of the
target function.

I now implemented that too, and now everything makes sense:

Traceback (most recent call last):
File "pseudoc_tool.py", line 91, in <module>
File "../xforms.py", line 25, in print
TypeError: unexpected keyword argument 'noann'


--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/VY3QAU2TEXOPELGDS7RTQOX4357QWILR/
Code of Conduct: http://python.org/psf/codeofconduct/

Terry Reedy

unread,
Feb 21, 2021, 5:10:51 PM2/21/21
to pytho...@python.org, python...@python.org
On 2/21/2021 12:04 PM, Paul Sokolovsky wrote:

> Traceback (most recent call last):
> File "pseudoc_tool.py", line 91, in <module>
> first_class_function_value(func, **pass_params)
> TypeError: print() got an unexpected keyword argument 'noann'

This is not typical behavior in current Python (3.8+).

def f(): pass
def g(): f(a=0)
g()

for instance, results in

Traceback (most recent call last):
File "F:\Python\a\tem3.py", line 3, in <module>
g()
File "F:\Python\a\tem3.py", line 2, in g
def g(): f(a=0)
TypeError: f() got an unexpected keyword argument 'a'

def f(): print(b=4)
def g(): f()
g()

gives me

Traceback (most recent call last):
File "F:\Python\a\tem3.py", line 3, in <module>
g()
File "F:\Python\a\tem3.py", line 2, in g
def g(): f()
File "F:\Python\a\tem3.py", line 1, in f
def f(): print(b=4)
TypeError: 'b' is an invalid keyword argument for print()

Can you create a minimal complete verifiable example and post it on
bugs.python.org. This is where bug reports belong, not on pydev and
python-ideas.

> I now implemented that too, and now everything makes sense:
>
> Traceback (most recent call last):
> File "pseudoc_tool.py", line 91, in <module>
> File "../xforms.py", line 25, in print
> TypeError: unexpected keyword argument 'noann'

Since you have a fix in your repository, perhaps you can include a PR
against cpython master branch.


--
Terry Jan Reedy
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/QK2FWYBY3SJ2KPMJ37HCEF7QM5DDN2MT/

Paul Sokolovsky

unread,
Feb 22, 2021, 5:19:45 AM2/22/21
to Peter Otten, Terry Reedy, pytho...@python.org, python...@python.org
Hello,

On Mon, 22 Feb 2021 10:44:19 +0100
Peter Otten <__pet...@web.de> wrote:

> On 21/02/2021 23:06, Terry Reedy wrote:
> > On 2/21/2021 12:04 PM, Paul Sokolovsky wrote:
> >
> >> Traceback (most recent call last):
> >>    File "pseudoc_tool.py", line 91, in <module>
> >>      first_class_function_value(func, **pass_params)
> >> TypeError: print() got an unexpected keyword argument 'noann'
> >
> > This is not typical behavior in current Python (3.8+).
>
> The way I understand it's not about print(), it's about
> disambiguating multiple functions with the same name.
> Example:
>
> PS > type .\ambiguous_names.py
> import random
>
> def do_stuff():
> pass
>
> f = do_stuff
>
> def do_stuff(a, b):
> pass
>
> g = do_stuff
>
> random.choice([f, g])(42)

Thanks, that's exactly what I meant, and a repro with random "roulette"
is also what I had in mind, I just didn't get to it yet ;-).

> PS > py .\ambiguous_names.py
> Traceback (most recent call last):
> File "...\ambiguous_names.py", line 13, in <module>
> random.choice([f, g])(42)
> TypeError: do_stuff() missing 1 required positional argument: 'b'
>
> The traceback gives no clue which of the two do_stuff() functions
> caused the error, you have to check both implementations.
>
> If that is a comman problem one might consider including module name
> and co_firstlineno in the message, or at least adding the relevant
> do_stuff() function to the exception's args.

As my original message argues, that's a workaround. Python tracebacks
already have places where they show source file and line number -
namely, the individual traceback entries. So, instead of cramming that
info into the exception message, there should be additional last (latest
in the order of execution) traceback entry, pointing to the exact
function which had parameter mismatch. As I mentioned, I implemented
that in my Python dialect, which happened to have exactly the same
problem (code is not based on CPython).

It looks like:

Traceback (most recent call last):
File "pseudoc_tool.py", line 91, in <module>
File ".../xforms.py", line 25, in print
TypeError: unexpected keyword argument 'noann'

- that makes clear that it's "print" function of "xforms.py" module,
line 25, which got an unexpected keyword argument.


--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/7SJTTAB7TZQYZ6WSRAAPSLLNMRF2OP7F/

Peter Otten

unread,
Feb 22, 2021, 11:45:51 AM2/22/21
to Terry Reedy, pytho...@python.org, python...@python.org
On 21/02/2021 23:06, Terry Reedy wrote:
> On 2/21/2021 12:04 PM, Paul Sokolovsky wrote:
>
>> Traceback (most recent call last):
>>    File "pseudoc_tool.py", line 91, in <module>
>>      first_class_function_value(func, **pass_params)
>> TypeError: print() got an unexpected keyword argument 'noann'
>
> This is not typical behavior in current Python (3.8+).

The way I understand it's not about print(), it's about
disambiguating multiple functions with the same name.
Example:

PS > type .\ambiguous_names.py
import random

def do_stuff():
pass

f = do_stuff

def do_stuff(a, b):
pass

g = do_stuff

random.choice([f, g])(42)


PS > py .\ambiguous_names.py
Traceback (most recent call last):
File "...\ambiguous_names.py", line 13, in <module>
random.choice([f, g])(42)
TypeError: do_stuff() missing 1 required positional argument: 'b'

The traceback gives no clue which of the two do_stuff() functions caused
the error, you have to check both implementations.

If that is a comman problem one might consider including module name and
co_firstlineno in the message, or at least adding the relevant
do_stuff() function to the exception's args.
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/E7OUHLWXP3NHEM3UZOBWP5AQRSFL5RDO/

Barry Scott

unread,
Feb 22, 2021, 2:51:24 PM2/22/21
to Paul Sokolovsky, Peter Otten, Terry Reedy, Python-Dev, python...@python.org


On 22 Feb 2021, at 10:15, Paul Sokolovsky <pmi...@gmail.com> wrote:

It looks like:

Traceback (most recent call last):
 File "pseudoc_tool.py", line 91, in <module>
 File ".../xforms.py", line 25, in print
TypeError: unexpected keyword argument 'noann'

- that makes clear that it's "print" function of "xforms.py" module,
line 25, which got an unexpected keyword argument.

You are proposing to fake a stack frame that I have to know is not a stack frame but is in fact the location of the function in the exception?
I'm -1 on that as its confusing.

Having checked that its python code and not a C extension function you could use the info
in fn.__code__ to get the filename and line of where the function is defined and put that info into the exception.

Example of the info:
| >>> os.path.join.__code__
<code object join at 0x7fb19aebc7c0, file "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/posixpath.py", line 71>

I use repr(fn.__code__) a lot when debugging complex code.

Barry

Paul Sokolovsky

unread,
Feb 22, 2021, 3:12:54 PM2/22/21
to Barry Scott, Peter Otten, Terry Reedy, Python-Dev, python...@python.org
Hello,

On Mon, 22 Feb 2021 19:47:04 +0000
Barry Scott <ba...@barrys-emacs.org> wrote:

> > On 22 Feb 2021, at 10:15, Paul Sokolovsky <pmi...@gmail.com> wrote:
> >
> > It looks like:
> >
> > Traceback (most recent call last):
> > File "pseudoc_tool.py", line 91, in <module>
> > File ".../xforms.py", line 25, in print
> > TypeError: unexpected keyword argument 'noann'
> >
> > - that makes clear that it's "print" function of "xforms.py" module,
> > line 25, which got an unexpected keyword argument.
>
> You are proposing to fake a stack frame that I have to know is not a
> stack frame but is in fact the location of the function in the
> exception?

No, I'm proposing to stop faking lack of the last stack frame due to
CPython's implementation details. See the original message for more
info.

> I'm -1 on that as its confusing.
>
> Having checked that its python code and not a C extension function
> you could use the info in fn.__code__ to get the filename and line of
> where the function is defined and put that info into the exception.

Could use crystal ball, even.

>
> Example of the info:
> | >>> os.path.join.__code__
> <code object join at 0x7fb19aebc7c0, file
> "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/posixpath.py",
> line 71>
>
> I use repr(fn.__code__) a lot when debugging complex code.
>
> Barry
>



--
Best regards,
Paul mailto:pmi...@gmail.com
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/FB65TFWGZEBENGT3W2X7RBPY63RINU25/
Reply all
Reply to author
Forward
0 new messages