Pls read: new directions for Leo

[Edward K. Ream]

In this post I'll discuss recent new ideas arising from my study of mypy. You could call this an Engineering Notebook post, but it should interest many Leonistas. The topics discussed here will likely keep me busy for years.

Executive summary

Text traces and python's pdb have been my main tools for studying mypy. But text traces, no matter how well designed, can't easily be analyzed further. Instead, an enhanced debugger could gather persistent python data structures.

Leo's existing clone-find commands represent relations between outline nodes. We can extend this idea! User-defined clone-find-by-relation (cfr) scripts can create outlines (relationships) using those python data structures.

An enhanced (Leo aware!) python debugger

Leo's g.SherlockTracer class is a python debugger that produces text traces. A similar debugger could create persistent python data structures.

User-defined scripts (in the debugger?) could then "mine" those data. Those scripts could even create  Leo outlines representing the data.

Representing data relationships in Leo outlines

clone-find-by-relation is a framework for invention, not single command. User-defined scripts will have the following general form:

- A data-gathering prepass gathers data from outside Leo, creating an input outline.

- User-defined cfr scripts call c.cloneFindByPredicate one or more times to create relationship outlines from the input outline. The predicates passed to c.cloneFindByPredicate can define any desired relationship!

- An optional postpass can beautify and simplify the relationship outlines.

Summary

clone-find-by-relation is a framework for invention, not a Leo command.

clone-find-by-relation can readily represent any mathematical relationship. Relationships are not confined to be points, strings, nodes, or anything else.

An enhanced debugger will gather dynamic data. In contrast, Leo's clone-find commands represent static relations between nodes.

I'll use the tools described here to study mypy. This will be an iterative process that will further refine the tools.

Edward

[Edward K. Ream]

On Thursday, June 16, 2022 at 7:56:22 AM UTC-5 Edward K. Ream wrote:

clone-find-by-relation is a framework for invention, not a Leo command.

...

An enhanced debugger will gather dynamic data. In contrast, Leo's clone-find commands represent static relations between nodes.

The ideas came to me in a different order than that shown. Something like this:

- A vague initial idea: I need better visualization tools.

- Aha: Neither traces nor any kind of picture would suffice.

- Aha: Leo outlines must be part of the visualization.

- Aha: Leo's clones can represent any (mathematical) relation on any data.

- Aha: A special-purpose debugger can collect arbitrary dynamic python data.

Edward

[Edward K. Ream]

On Thursday, June 16, 2022 at 9:23:50 AM UTC-5 Edward K. Ream wrote:

- Aha: A special-purpose debugger can collect arbitrary dynamic python data.

This supposed Aha looks more like a brain spike :-) I have no idea why I thought a special purpose debugger might help gather data.

The easy way to collect python data is with added python statements in the source code that I'm studying.  In contrast, using a debugger to gather data would be roundabout, slow and obscure.

Perhaps I was thinking that an enhanced g.SherlockTracer class could gather python data, not text traces, but I if so I was almost certainly jumping to conclusions.

Edward

[tbp1...@gmail.com]

Leaving aside a possible customized debugger, is this the concept?

1. Some utility, possibly a debugger, possibly special trace statements, outputs information directly into a Leo outline (I would prefer that it not be the outline under test);

2. Use clone-xx with predicates on this outline to extract nodes of interest that may be related;

Personally, I think these capabilities would be very helpful:

1. When a stack trace outputs a line/position in a file (which could be, say, a Python standard library module, not necessarily your own code), control-click would open that file for you and navigate to the location;

2. Have the debugger highlight the current line in its actual node in its Leo outline, instead of just giving the line number in the generated script-file.

3. Having some convenient way to monkey patch a routine to add traces and print statements temporarily without actually modifying the original outline.  This is less important because one can always use @Edward's approach of modifying the code but running another Leo session to test it (or, if running it as an external file, you can just run it) so at least you don't have to keep closing and reopening Leo each time you make a change.

[Edward K. Ream]

On Fri, Jun 17, 2022 at 8:40 AM tbp1...@gmail.com <tbp1...@gmail.com> wrote:

Leaving aside a possible customized debugger, is this the concept?

1. Some utility...outputs information directly into a Leo outline (I would prefer that it not be the outline under test);

The utility would be regular assignment statements into a study database (dict). At the end of the study run, code could convert the study database to a Leo outline.

2. Use clone-xx with predicates on this outline to extract nodes of interest that may be related;

Yes, that's the general idea.

Personally, I think these capabilities would be very helpful:

1. When a stack trace outputs a line/position in a file (which could be, say, a Python standard library module, not necessarily your own code), control-click would open that file for you and navigate to the location;

I wasn't thinking along those lines. When the data are in an outline, we can use clones to filter and reorganize the nodes.

Edward

[tbp1...@gmail.com]

I switched subjects a bit but I see I wasn't clear enough.,  

Personally, I think these capabilities would be very helpful:

1. When a stack trace outputs a line/position in a file (which could be, say, a Python standard library module, not necessarily your own code), control-click would open that file for you and navigate to the location;

I wasn't thinking along those lines. When the data are in an outline, we can use clones to filter and reorganize the nodes.

Edward

I was talking about when we get a stack trace during execution of some code that's in a Leo outline.  The trace shows a number of steps with file names and line numbers.  If a step occurs in a file that is in a Leo outline, it would be useful to be able to go directly to the particular line in right node in the outline.  If the step occurs in a non-Leo file, it would be helpful to open that file and go directly to the right line.  I realize this isn't quite on-topic, but the subject got me thinking about other ways to make study and debugging easier and more effective.

[Edward K. Ream]

On Sat, Jun 18, 2022 at 10:37 AM tbp1...@gmail.com <tbp1...@gmail.com> wrote:

I was talking about when we get a stack trace during execution of some code that's in a Leo outline.  The trace shows a number of steps with file names and line numbers.  If a step occurs in a file that is in a Leo outline, it would be useful to be able to go directly to the particular line in right node in the outline.  If the step occurs in a non-Leo file, it would be helpful to open that file and go directly to the right line.  I realize this isn't quite on-topic, but the subject got me thinking about other ways to make study and debugging easier and more effective.

These enhancements might be useful, but they wouldn't change my life :-)

Edward