ENB: About mypy

[Edward K. Ream]

Issues #2049 and #2103 concern mypy and static type checking. I worked yesterday on these issues in the ekr-2103 branch.  This Engineering Notebook post "thinks out loud" re these issues. Feel free to ignore.

Significance

The improved mypy command shows errors as clickable links in Leo's log pane. That part of the work is complete. Otherwise, completing these issues is of little importance to Leo.

Still, I am eager to complete these issues. At that point, Leo 6.4 will be almost complete!

I am near the end of my interest and involvement in static type checking. Imo, static type checking adds little to python. If you want proper type checking, try rust or other typed languages.

Experience shows that mypy complaints are generally less useful than pylint complaints. Altering code to fix mypy complaints is always risky.

Completing these issues

Yesterday's work showed that some strategy is required to do what I want and no more. Here is the general plan:

Step 1: Create a new branch, say ekr-mypy2, to work on the mypy command itself. The main question is what to do about mypy arguments. At present, the mypy command supports the `@data mypy-arguments` and the `@int mypy-link-limit` settings. However, it will probably be better always to use the .mypy.ini file. We shall see.

Step 2: I'll merge ekr-mypy2 into devel, then create another new branch, say ekr-stubs, to work on annotating leoGlobals.py. Much of this work is complete, but I was pretty hasty yesterday, so I may want to redo some work as a precaution against bugs.   leoGlobals.pyi might be the only "deliverable" of this branch. We shall see.

Step 3: mypy does seem to have found problems with the following methods: execute_single_pytest,  writeZipFile, and (maybe)  indentToCommentColumn. But it would be unbearable to introduce new bugs while trying to placate mypy! I'll investigate these problems and create separate issues (and PR's) as needed for each.

Summary

This work will mark the end of my interest and involvement with mypy and static type checking for Leo.

Impatience is the clear enemy here. The work is mundane, but I must not introduce any new bugs!

Edward

[tbp1...@gmail.com]

On Thursday, July 29, 2021 at 9:41:53 AM UTC-4 Edward K. Ream wrote:

Step 1: Create a new branch, say ekr-mypy2, to work on the mypy command itself. The main question is what to do about mypy arguments. At present, the mypy command supports the `@data mypy-arguments` and the `@int mypy-link-limit` settings. However, it will probably be better always to use the .mypy.ini file. We shall see.

If you go the .ini route, please get Leo to open the ini file for editing, or at least tell us where it is.  It's always a pain to try to figure out where the right ini file is for a program, then find it.  For example, I never remember where the right config files is for pylint.  Leo should be able to know, letting us skip all that.

[Edward K. Ream]

On Thu, Jul 29, 2021 at 10:02 AM tbp1...@gmail.com <tbp1...@gmail.com> wrote:

> If you go the .ini route, please get Leo to open the ini file for editing, or at least tell us where it is. 

Good idea. Will do.

Edward

[Edward K. Ream]

An update.

> Imo, static type checking adds little to python. If you want proper type checking, try rust or other typed languages.

That's a hasty verdict. mypy runs off of annotations, so it's unfair to expect mypy to be much help without them!

The annotations of the types returned from functions and methods are likely to be the most important. Such annotations cut the Gordian knot of type inferences. Local inferences are straightforward; inferring return types is much harder.

> Impatience is the clear enemy here. The work is mundane, but I must not introduce any new bugs!

Yesterday I completed the annotations for leoGlobals.py so that mypy now passes launchLeo.py.

The work was indeed tedious. Afterward, I thought that a wax-on script, the companion of wax-off, would have saved me much of the tedium. But wait. Adding annotations for returned types is much harder than adding annotations for arguments, and wax-on would not help.

Oh my. As I write this, I think I remember that my old make_stub_files (msb) project calculates returned types based on pattern matches of the values of all "return" statements. Aha! Time to experiment with msb!

Summary

Aha! My old make_stub_files project may be more valuable than I remembered. I'll delay writing the wax-on script.

My focus for Leo 6.4 is to complete Leo's mypy command. I'll also carefully fix any bugs that mypy reveals.

Except for leoGlobals.pyi, I will create no new stub files for Leo 6.4. I will not add annotations to Leo's source files themselves.

Edward

[Ville M. Vainio]

FWIW, I noted that mypy is much better when run in --strict mode. Otherwise, mypy will ignore some clear errors.

Also, there are some tools that can automate creating mypy annotations: https://mypy.readthedocs.io/en/stable/existing_code.html#automate-annotation-of-legacy-code

--
You received this message because you are subscribed to the Google Groups "leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/20c35a7e-d6e1-40c5-ba8c-84561d1969d1n%40googlegroups.com.

[Edward K. Ream]

On Fri, Jul 30, 2021 at 11:07 AM Ville M. Vainio <viva...@gmail.com> wrote:

FWIW, I noted that mypy is much better when run in --strict mode. Otherwise, mypy will ignore some clear errors.

Also, there are some tools that can automate creating mypy annotations: https://mypy.readthedocs.io/en/stable/existing_code.html#automate-annotation-of-legacy-code

Good to hear from you. Thanks for these links.

Edward

[Edward K. Ream]

On Friday, July 30, 2021 at 11:07:49 AM UTC-5 Ville M. Vainio wrote:

> there are some tools that can automate creating mypy annotations

I have just created #2106 for this, assigned to Leo 6.5. It's too late in the 6.4 cycle to make wholesale changes to code.

Edward

[Edward K. Ream]

On Thursday, July 29, 2021 at 8:41:53 AM UTC-5 Edward K. Ream wrote:

Issues #2049 and #2103 concern mypy and static type checking.

I have just merged #2103 into devel. Eventually almost all the changes to leoGlobals.py moved into the stub file, leaving only a few suppressions. This will be the last major change to Leo's code base before Leo 6.4.

#2049 is still open. However, the first round of changes has been merged into devel. I expect to complete #2049 in a day or two.

Félix and I are collaborating on a possible improvement to leoserver.py. This work will have no effect on the rest of Leo's code base.

I have just created #2106 for future mypy work. That work will be done for Leo 6.5.

Edward