ENB: about python-to-typescript

[Edward K. Ream]

In this Engineering Notebook post I'll summarize the work so far on converting Leo's core sources to typescript. This work supports Félix's leojs project.

Speed and accuracy

From the very beginning I knew that the python-to-typescript command (script) would save lots of time. I estimated that an hour's work on the script would be worthwhile if it saved just a minute of time massaging the resulting .ts file.

Aha! Just now I see that the script is worth any amount of time because the script eliminates potential editing errors!

Iterative workflow

The momentum of the project is increasing. I convert a file (leoAtFile.py) to leoAtFile.ts, copy the file to the leojs repo, and use vs-code to report errors. I may correct some errors by hand, just to make sure that I understand what is going on, but all such corrections are ephemeral. I go back to Leo and improve python-to-typescript and repeat the process!

@button cvt-at-file

The following script allows me to change the python-to-typescript command without reloading Leo:

g.cls()
import importlib
import leo.commands.convertCommands as convertCommands
importlib.reload(convertCommands)
h = '@file leoAtFile.py'
root = g.findNodeAnywhere(c, h)
assert root, h
x = convertCommands.ConvertCommandsClass(c).PythonToTypescript(c, alias='at')
x.convert(root)

This @button node is customized to convert leoAtFile.py, so all I need to do is <Alt-x>cv<tab><return> Doing converts the file and puts the result as the last top-level node of my outline.

Note: The @button script should not use c.convertCommands, because that a reload wouldn't change the already-created object. Instead, the @buttons script instantiates a new ConvertCommandsClass object, which in effect reloads the PythonToTypescript class. This is an important technique for Leo's devs to know.

The main loop

I consider myself fortunate to have "stumbled" upon the overall structure of py2ts.convert_body. This method is the heart of the command.  You could say the main loop in this method was a "lucky guess". I won't show you the code, but it's worth a look.

The general idea is to split p.b into a mutable list of lines, and then apply regex's, one by one, to each line until one pattern matches. A dispatch table contains a list of (regex, handler) tuples. The order of tuples in the dispatch table sometimes matters.

The main loop calls the corresponding handler when a regex matches. Each handle alters the list of lines and returns the index of the next line to be considered.

I got lucky with this organization because each handler turned out to be incredibly elegant!  Here is the handler for 'if' statements:

if_pat = re.compile(r'^([ \t]*)if[ \t]+(.*?):(.*?)\n')

def do_if(self, i, lines, m, p):
    j = self.find_indented_block(i, lines, m, p)
    lws, cond, tail = m.group(1), m.group(2).strip(), m.group(3).strip()
    cond_s = cond if cond.startswith('(') else f"({cond})"
    tail_s = f" // {tail}" if tail else ''
    lines[i] = f"{lws}if {cond_s} {{{tail_s}\n"
    lines.insert(j, f"{lws}}}\n")
    return i + 1  # Advance.

The find_indented_block method returns the index into lines of the end of the if block. The handler changes the "if line" in place and inserts a line containing an '}' (properly indented!) at the end of the block.  The dispatch table has an entry:

(self.if_pat, self, do_if)

As you can see it's more convenient to define "if_pat" in the node that defines the do_if handler.

The simplest organization possible

convert_body applies various global regexes to p.b before running the main loop. These regexes use the re.MULTILINE flag to apply a pattern to each line of p.b.

The question arises, would it be simpler to use multiline regexes everywhere? The short answer is, "No!":

1. Although testing the dispatch dict for each line of p.b would seem to be slow, speed doesn't matter. The command works almost instantaneously.

2. As shown above, each handler is super elegant. For example, it's trivial to insert closing brackets into the list of lines. Using, say, re.sub would be significantly more complicated.

Summary

Leo's evolving python-to-typescript command is my entry into the typescript world.

This command is worth any amount of work. It eliminates tedious (error-prone!) manual text munging.

I feed leoAtFile.ts (the results of the @button script) into vs-code, which then points out all the remaining problems. I use those problems to improve the script. I'll repeat this process until any remaining problems would be too difficult to fix in a script.

This whole process is driving my learning of typescript. My energy levels are off the charts!

Edward

[tbp1...@gmail.com]

After your work on ASTs, I would have thought you might try generating typescript from them.

[Edward K. Ream]

On Mon, Oct 18, 2021 at 1:37 PM tbp1...@gmail.com <tbp1...@gmail.com> wrote:

After your work on ASTs, I would have thought you might try generating typescript from them.

I never thought of doing that. I might consider doing so if real accuracy were important, but the present approach is simple, good, and flexible.

Edward