ENB: Progress re mypy and make_stub_files

[Edward K. Ream]

For the last few days, I have been adding type annotations for mypy. mypy has gone from being mysterious to just another tool.

This Engineering Notebook post discusses what I have learned and what remains to do.

tl;dr: Read the summary

Settings

At first, mypy's operation seemed mysterious to me. Using the proper settings helped a lot.

The following settings appear in .mypy.ini in the ekr-annotate branch. They will eventually migrate into devel:

check_untyped_defs = True

By default, this setting is False, which means that mypy only checks functions and methods that contain type annotations. No wonder it didn't seem like mypy was doing anything :-)

strict_optional = False

If False, mypy treats None as compatible with every type. False by default in mypy versions earlier than 0.600. This option eliminates a lot of spurious warnings.

incremental = True

True is the default, and there seems no reason to change it.  When True, mypy caches type data.

cache_dir = mypy_stubs

Leo's .gitignore file contains mypy_stubs/, so git will never this directory. Imo, this is as it should be.

show_error_codes = True

This setting shows the error codes in mypy complaints. You can suppress only a particular error code by adding the # type:ignore[some-error-code].

follow_imports = skip

By default (in .mypy.ini) this setting is True. However, setting this to skip is useful for Qt wrapper files.

Using mypy

Using mypy is straightforward, provided check_untyped_defs is True and strict_optional is False :-)  With these settings, mypy works much like pylint.

To check all of Leo, it suffices to do:

mypy launchLeo.py

To limit checking to a particular file, say leoAst.py, do:

mypy leo\core\leoApp.py --follow-imports=skip

As you will soon learn, mypy requires annotations for many dicts and lists. mypy typically does not need help inferring the types of other local vars in functions and methods.

To increase the strength of type checking, one should add annotations for signatures. 

To annotate p, do something this:

from leo.core import leoNodes
Pos = "leoNodes.Position"

...

def f1(self, p: Pos):

To annotate c, do something like this:

from leo.core.leoCommands import Commands as Cmdr

...

def f2(self, c: Cmdr):

It's easy to annotate Leo's convention for s, i, etc.

def f3(self, s: str, i: int):

Stub files

Imo, stub files have outlived their usefulness. It's simpler and more explicit to use annotations in Leo. For this reason, I plan to wind down my make_stub_files project.

I have abandoned the effort to create a mypy plugin to make Leo's conventions available directly to mypy. I have updated #1944 accordingly.

Summary

.mypy.ini contains default settings. Learning which defaults to use made mypy much less mysterious. Type annotations allow mypy to find errors that pylint can't find.

After an initial learning period, mypy is about as easy to use as pylint:

- mypy supports gradual type checking. We can add annotations gradually.

- It's easy to make Leo's naming conventions available to mypy.

- Imo, stub files are no longer useful in the python 3 world. I shall wind down my make_stub_files project.

All comments and questions are welcome.

Edward

[Edward K. Ream]

On Friday, June 25, 2021 at 5:53:12 AM UTC-5 Edward K. Ream wrote:

For the last few days, I have been adding type annotations for mypy. mypy has gone from being mysterious to just another tool.

Please don't panic. I have no intention of adding type hints to all of Leo's files.

I have added type hints to leoNodes.py, and will add more type hints to leoAst.py. That's the extent of my present plans.

Edward