ENB: About the check_leo script

[Edward K. Ream]

This Engineering Notebook post discusses leo/scripts/check_leo.py and my plans for future work. See #4483 and PR #4484 for details.

Unexpected success

​The script is unexpectedly simple and powerful.  Aha! At last I see why: the script doesn't need to understand the context of any ast node! Why? Because:

    The script leverages Leo's naming conventions!

​These conventions apply (or should apply) everywhere, regardless of context.

Liberation

For years, I've struggled to understand complex apps like mypy and pylint. But check_leo takes a much simpler approach. There is no need to study complex algorithms in other apps! The script's pragmatic approach has found bugs that neither mypy nor pylint found. That's plenty good enough.

Next steps

At present, the script (visitor.Attribute) checks only attributes of the form x.y, where x is a name and y is a string. I'll soon extend those checks to chains of the form n1.n2.n3...nN.y. The script will print out a sorted list of all unique chains.

Depending on what I see, I'll probably add more live objects and add more checks to the Attribute visitor. I'll also add files in leo/commands and the most important files in leo/plugins to the list of files to be checked.

Speed

Here are typical timing stats:

49 files
Setup: 1.02 sec.
 Scan: 0.78 sec.
Total: 1.80 sec

​Setup is the time to init Leo's bridge, an essential part of creating live objects.

The takeaway: improving the speed of the script is completely unimportant. For now, using live (Leo) objects simplifies everything.

Static analysis

The limitations of check_leo.py are obvious: The script is Leo-specific and relies on Leo's bridge to create live objects.

Instead, one could imagine a script that statically analyzes classes throughout the program. #4483 contains a preliminary design. The script would process files using multiple passes.

There are subtle interactions between scanning for imports and scanning for classes. All details are fuzzy now. A third pass might create a database of classes, and a last pass would check attributes using that database, or live summary objects. All passes would be extremely fast, so speed would not be an issue.

The real question is how much context would be needed in each pass. Only a context-free algorithm has much chance of being useful for programs other than Leo.

Summary

check_leo.py is already a great success. It will soon check all of Leo's attribute chains. Leo's naming conventions allow the script to ignore all context.

I'll probably experiment with using static analysis in check_leo.py. This experiment will fail unless the code can remain (mostly) context-free. Otherwise, the code's complexity would explode.

All comments are welcome.

Edward